{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interpreting Nodes and Edges by Saliency Maps in GAT\n",
    "\n",
    "This demo shows how to use integrated gradients in graph attention networks to obtain accurate importance estimations for both the nodes and edges. The notebook consists of three parts:\n",
    "\n",
    "setting up the node classification problem for Cora citation network\n",
    "training and evaluating a GAT model for node classification\n",
    "calculating node and edge importances for model's predictions of query (\"target\") nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy import stats\n",
    "import os\n",
    "import time\n",
    "import sys\n",
    "import stellargraph as sg\n",
    "from copy import deepcopy\n",
    "\n",
    "\n",
    "from stellargraph.mapper import FullBatchNodeGenerator\n",
    "from stellargraph.layer import GAT, GraphAttention\n",
    "\n",
    "from tensorflow.keras import layers, optimizers, losses, metrics, models, Model\n",
    "from sklearn import preprocessing, feature_extraction, model_selection\n",
    "from tensorflow.keras import backend as K\n",
    "import matplotlib.pyplot as plt\n",
    "from stellargraph import datasets\n",
    "from IPython.display import display, HTML\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the CORA network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "The Cora dataset consists of 2708 scientific publications classified into one of seven classes. The citation network consists of 5429 links. Each publication in the dataset is described by a 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary. The dictionary consists of 1433 unique words."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset = datasets.Cora()\n",
    "display(HTML(dataset.description))\n",
    "dataset.download()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the graph from edgelist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "edgelist = pd.read_csv(\n",
    "    os.path.join(dataset.data_directory, \"cora.cites\"),\n",
    "    header=None,\n",
    "    names=[\"source\", \"target\"],\n",
    "    sep=\"\\t\",\n",
    ")\n",
    "edgelist[\"label\"] = \"cites\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "Gnx = nx.from_pandas_edgelist(edgelist, edge_attr=\"label\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "nx.set_node_attributes(Gnx, \"paper\", \"label\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the features and subject for the nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = [\"w_{}\".format(ii) for ii in range(1433)]\n",
    "column_names = feature_names + [\"subject\"]\n",
    "node_data = pd.read_csv(\n",
    "    os.path.join(dataset.data_directory, \"cora.content\"),\n",
    "    header=None,\n",
    "    names=column_names,\n",
    "    sep=\"\\t\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For machine learning we want to take a subset of the nodes for training, and use the rest for validation and testing. We'll use scikit-learn again to do this.\n",
    "\n",
    "Here we're taking 140 node labels for training, 500 for validation, and the rest for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data, test_data = model_selection.train_test_split(\n",
    "    node_data, train_size=140, test_size=None, stratify=node_data[\"subject\"]\n",
    ")\n",
    "val_data, test_data = model_selection.train_test_split(\n",
    "    test_data, train_size=500, test_size=None, stratify=test_data[\"subject\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({'Neural_Networks': 42,\n",
       "         'Theory': 18,\n",
       "         'Probabilistic_Methods': 22,\n",
       "         'Reinforcement_Learning': 11,\n",
       "         'Genetic_Algorithms': 22,\n",
       "         'Case_Based': 16,\n",
       "         'Rule_Learning': 9})"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import Counter\n",
    "\n",
    "Counter(train_data[\"subject\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting to numeric arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our categorical target, we will use one-hot vectors that will be fed into a soft-max Keras layer during training. To do this conversion ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_encoding = feature_extraction.DictVectorizer(sparse=False)\n",
    "\n",
    "train_targets = target_encoding.fit_transform(train_data[[\"subject\"]].to_dict(\"records\"))\n",
    "val_targets = target_encoding.transform(val_data[[\"subject\"]].to_dict(\"records\"))\n",
    "test_targets = target_encoding.transform(test_data[[\"subject\"]].to_dict(\"records\"))\n",
    "\n",
    "node_ids = node_data.index\n",
    "all_targets = target_encoding.transform(node_data[[\"subject\"]].to_dict(\"records\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now do the same for the node attributes we want to use to predict the subject. These are the feature vectors that the Keras model will use as input. The CORA dataset contains attributes 'w_x' that correspond to words found in that publication. If a word occurs more than once in a publication the relevant attribute will be set to one, otherwise it will be zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "node_features = node_data[feature_names]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating the GAT model in Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now create a StellarGraph object from the NetworkX graph and the node features and targets. It is StellarGraph objects that we use in this library to perform machine learning tasks on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NetworkXStellarGraph: Undirected multigraph\n",
      " Nodes: 2708, Edges: 5278\n",
      "\n",
      " Node types:\n",
      "  paper: [2708]\n",
      "    Edge types: paper-cites->paper\n",
      "\n",
      " Edge types:\n",
      "    paper-cites->paper: [5278]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "G = sg.StellarGraph(Gnx, node_features=node_features)\n",
    "print(G.info())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To feed data from the graph to the Keras model we need a generator. Since GAT is a full-batch model, we use the `FullBatchNodeGenerator` class to feed node features and graph adjacency matrix to the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "generator = FullBatchNodeGenerator(G, method=\"gat\", sparse=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For training we map only the training nodes returned from our splitter and the target values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_gen = generator.flow(train_data.index, train_targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can specify our machine learning model, we need a few more parameters for this:\n",
    "\n",
    " * the `layer_sizes` is a list of hidden feature sizes of each layer in the model. In this example we use two GAT layers with 8-dimensional hidden node features at each layer.\n",
    " * `attn_heads` is the number of attention heads in all but the last GAT layer in the model\n",
    " * `activations` is a list of activations applied to each layer's output\n",
    " * Arguments such as `bias`, `in_dropout`, `attn_dropout` are internal parameters of the model, execute `?GAT` for details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To follow the GAT model architecture used for Cora dataset in the original paper [Graph Attention Networks. P. Velickovic et al. ICLR 2018 https://arxiv.org/abs/1803.07294], let's build a 2-layer GAT model, with the 2nd layer being the classifier that predicts paper subject: it thus should have the output size of `train_targets.shape[1]` (7 subjects) and a softmax activation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "gat = GAT(\n",
    "    layer_sizes=[8, train_targets.shape[1]],\n",
    "    attn_heads=8,\n",
    "    generator=generator,\n",
    "    bias=True,\n",
    "    in_dropout=0,\n",
    "    attn_dropout=0,\n",
    "    activations=[\"elu\", \"softmax\"],\n",
    "    normalize=None,\n",
    "    saliency_map_support=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expose the input and output tensors of the GAT model for node prediction, via GAT.build() method:\n",
    "x_inp, predictions = gat.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create the actual Keras model with the input tensors `x_inp` and output tensors being the predictions `predictions` from the final dense layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Model(inputs=x_inp, outputs=predictions)\n",
    "model.compile(\n",
    "    optimizer=optimizers.Adam(lr=0.005),\n",
    "    loss=losses.categorical_crossentropy,\n",
    "    weighted_metrics=[\"acc\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model, keeping track of its loss and accuracy on the training set, and its generalisation performance on the validation set (we need to create another generator over the validation data for this)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "val_gen = generator.flow(val_data.index, val_targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "1/1 - 8s - loss: 1.9606 - acc: 0.0643 - val_loss: 1.7862 - val_acc: 0.4460\n",
      "Epoch 2/10\n",
      "1/1 - 2s - loss: 1.6883 - acc: 0.6143 - val_loss: 1.6243 - val_acc: 0.4560\n",
      "Epoch 3/10\n",
      "1/1 - 2s - loss: 1.4529 - acc: 0.6714 - val_loss: 1.4871 - val_acc: 0.4820\n",
      "Epoch 4/10\n",
      "1/1 - 2s - loss: 1.2471 - acc: 0.7357 - val_loss: 1.3663 - val_acc: 0.5320\n",
      "Epoch 5/10\n",
      "1/1 - 2s - loss: 1.0648 - acc: 0.7571 - val_loss: 1.2545 - val_acc: 0.6060\n",
      "Epoch 6/10\n",
      "1/1 - 2s - loss: 0.9006 - acc: 0.8143 - val_loss: 1.1488 - val_acc: 0.6660\n",
      "Epoch 7/10\n",
      "1/1 - 2s - loss: 0.7531 - acc: 0.8714 - val_loss: 1.0507 - val_acc: 0.7060\n",
      "Epoch 8/10\n",
      "1/1 - 2s - loss: 0.6233 - acc: 0.9000 - val_loss: 0.9629 - val_acc: 0.7440\n",
      "Epoch 9/10\n",
      "1/1 - 2s - loss: 0.5118 - acc: 0.9214 - val_loss: 0.8870 - val_acc: 0.7840\n",
      "Epoch 10/10\n",
      "1/1 - 2s - loss: 0.4179 - acc: 0.9357 - val_loss: 0.8235 - val_acc: 0.8000\n"
     ]
    }
   ],
   "source": [
    "N = len(node_ids)\n",
    "history = model.fit_generator(\n",
    "    train_gen, validation_data=val_gen, shuffle=False, epochs=10, verbose=2\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def remove_prefix(text, prefix):\n",
    "    return text[text.startswith(prefix) and len(prefix):]\n",
    "\n",
    "def plot_history(history):\n",
    "    metrics = sorted(set([remove_prefix(m, \"val_\") for m in list(history.history.keys())]))\n",
    "    for m in metrics:\n",
    "        # summarize history for metric m\n",
    "        plt.plot(history.history[m])\n",
    "        plt.plot(history.history['val_' + m])\n",
    "        plt.title(m)\n",
    "        plt.ylabel(m)\n",
    "        plt.xlabel('epoch')\n",
    "        plt.legend(['train', 'validation'], loc='best')\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de3xV9Z3v/9cn9wtJCAm3XCDhJiHciYgiFMVatRVvFW21U52pntPL2HbmzBxmph09nekcZ+r02Pn9nM7Ynto5bW2LWi+nY2tvKAneCCrITQMkkEAg2SH3e7K/54+1IQEDBs3OSvZ+Px8PHmSvvbL5ZAPf99rftdbna845REQkesX4XYCIiPhLQSAiEuUUBCIiUU5BICIS5RQEIiJRTkEgIhLlFAQiIlFOQSAiEuUUBCIiUU5BIDJMZrbJzA6aWauZ7TWzmwY9d4+Z7Rv03PLQ9nwz+4WZ1ZtZg5n9//79BCJDi/O7AJFx5CCwBjgO3Ar82MzmAJcDDwA3AuXAbKDXzGKBXwJ/AD4D9AMlo1+2yPmZeg2JfDBm9hZwP/AF4Hnn3HfOev5S4DlgunOuz4cSRYZFU0Miw2Rmf2Rmb5lZk5k1AQuBbCAf79PC2fKBwwoBGes0NSQyDGY2E/gesB54xTnXH/pEYEA13nTQ2aqBGWYWpzCQsUyfCESGJxVwQD2Amd2N94kA4PvAfzOzFeaZEwqO14Fa4EEzSzWzJDNb7UfxIuejIBAZBufcXuCfgVeAE8AiYFvouSeAbwKPA63AM8Ak51w/cD0wBzgC1AC3jXrxIu9DJ4tFRKKcPhGIiEQ5BYGISJRTEIiIRDkFgYhIlBt39xFkZ2e7goICv8sQERlXduzYEXDOTR7quXEXBAUFBZSXl/tdhojIuGJmh8/1nKaGRESinIJARCTKKQhERKLcuDtHMJTe3l5qamro6uryu5SIkJSURF5eHvHx8X6XIiKjICKCoKamhrS0NAoKCjAzv8sZ15xzNDQ0UFNTQ2Fhod/liMgoiIipoa6uLrKyshQCI8DMyMrK0qcrkSgSEUEAKARGkN5LkegSEVNDIiKRpKcvSKCtm7rWbupbu6lr7aK+tZsr509hcd7EEf/zFAQjoKmpiccff5wvfOELF/R91113HY8//jgTJ478X6yIjC3OOVq6+s4Y2OtPD/RnDviNHb1DvkbWhEQFwVjV1NTEv/7rv74nCPr6+oiLO/db/Pzzz4e7NBEJs77+IA3tPdS1dFPf1uX9HhrcTw3spwb67r7ge74/IS6GyRMSmZKeSEFWKisLJzElLYnJaYlMSUsM/Z5E1oQE4mPDM5uvIBgBmzZt4uDBgyxdupT4+HiSkpLIzMxk//79vPvuu9x4441UV1fT1dXFl7/8Ze69915goF1GW1sb1157LZdffjkvv/wyubm5PPvssyQnJ/v8k4lEL+ccDe09HG5op7Y5NMC3dQ/6vYtAWzcN7T0Mtb5XRnL86YG8ZGYmU9KTTg/4A78nkZ4c5/t5uYgLgv/xf/ew91jLiL7mgpx07r+++JzPP/jgg+zevZu33nqLF198kY9//OPs3r379OWXP/jBD5g0aRKdnZ1cfPHF3HLLLWRlZZ3xGhUVFfz0pz/le9/7Hhs3buSpp57izjvvHNGfQ0TO5JyjsaOXykA7VYF2qhraqQy0c7ihg6pAO63dfWfsHxdjTA4N7nmZySybkXnWkXvi6ecT42J9+qkuXMQFwViwcuXKM67B/5d/+ReefvppAKqrq6moqHhPEBQWFrJ06VIAVqxYQVVV1ajVKxLpGtt7qGxo53BDO5WBjtODflWgnZaugcE+xiA3M5mCrFRuWp5LQVYqBdkp5ExMZkpaEhOT44mJibyr6iIuCM535D5aUlNTT3/94osv8rvf/Y5XXnmFlJQU1q1bN+Q1+omJiae/jo2NpbOzc1RqFYkUzR29VIYGd++ovp3K0JF9c+fAyVczyJ3oDfYbluZQkJVKYXYqBdmp5GUmj6sj+ZEScUHgh7S0NFpbW4d8rrm5mczMTFJSUti/fz+vvvrqKFcnEjlaunpPD/RVgY5BUzntZ1xpYwY5GckUZKfwicXTKcxOZWZWKoXZKeRPSonKwf58FAQjICsri9WrV7Nw4UKSk5OZOnXq6eeuueYa/u3f/o2ioiIuuugiVq1a5WOlImOfc47qk53srW2m4kTb6aP8qoYOTrb3nLFvTkYSM7NSuWbhdAqzU04f3edPSiEpXoP9cJkb6nT3GFZSUuLOXphm3759FBUV+VRRZNJ7KqOhrz/Iwfp29hxrZs+xFvYca2bvsZYz5u2npSdRkJ1y+qj+1GA/M0uD/YUwsx3OuZKhntMnAhEZFZ09/ew/3hIa8FvYe6yZ/cdbT19bnxgXw/zp6XxiSQ7FOekU52Qwb+oEUhI0TIWb3mERGXHNHb1nHOXvOdbCwfo2gqEJiPSkOIpzMvjMqpkU53qD/qzsVOLCdMOUnJ+CQEQ+MOccJ1q63zPo1zQOXPU2LT2J4px0rl04jQU5GRTnpJOXmez7TVQyQEEgIsMSDDqqGtpPT+2cms9vGHQCtzA7lSX5E/n0JTNYmJPBgpx0sicknudVZSxQEIjIe/T0BXn3RCt7Bx3l76ttob2nH4D4WGPulDSunD/Fm8/PzaBoejoTEjWkjEf6WxMRALr7+vn9vjo2l1ez7UCA3n5vQj8lIZYF09P55Io8ikNH+fOmppEQp/n8SKEg8MGECRNoa2vj2LFj3HfffTz55JPv2WfdunU89NBDlJQMebUXAA8//DD33nsvKSkpgNpaywezr7aFzeXVPPPmURo7epmWnsRnLy1gSf5EinPSKchKjci2CjJAQeCjnJycIUNguB5++GHuvPPO00GgttYyXM2dvTz31lE2l9fw9tFm4mONqxdM49aSPNbMnUysBv6oos92I2DTpk088sgjpx8/8MAD/P3f/z3r169n+fLlLFq0iGefffY931dVVcXChQsB6Ozs5Pbbb6eoqIibbrrpjF5Dn//85ykpKaG4uJj7778f8BrZHTt2jCuuuIIrrrgC8NpaBwIBAL797W+zcOFCFi5cyMMPP3z6zysqKuKee+6huLiYq6++Wj2Nokgw6Nh2IMCXf/YmK7/5O77+7B56+4Pcf/0CXvvrq3jkjuWsu2iKQiAKRd4ngl9tguNvj+xrTlsE1z54zqdvu+02vvKVr/DFL34RgM2bN/PCCy9w3333kZ6eTiAQYNWqVWzYsOGcl8x997vfJSUlhX379rFr1y6WL19++rlvfvObTJo0if7+ftavX8+uXbu47777+Pa3v82WLVvIzs4+47V27NjBY489xmuvvYZzjksuuYSPfOQjZGZmqt11FKpp7ODJHTU8UV7D0aZO0pPiuO3ifDaW5FOck67LOCUCg8AHy5Yto66ujmPHjlFfX09mZibTpk3jq1/9Klu3biUmJoajR49y4sQJpk2bNuRrbN26lfvuuw+AxYsXs3jx4tPPbd68mUcffZS+vj5qa2vZu3fvGc+fraysjJtuuul0F9Sbb76Z0tJSNmzYoHbXUaKrt58X9hznifIath0M4BxcPiebv7zmIj5WPE2tGeQMkRcE5zlyD6dbb72VJ598kuPHj3Pbbbfxk5/8hPr6enbs2EF8fDwFBQVDtp9+P5WVlTz00ENs376dzMxM7rrrrg/0Oqeo3XXkcs6x59jAid+Wrj5yJybz5fVzuWV5HvmTUvwuUcaoyAsCn9x2223cc889BAIBXnrpJTZv3syUKVOIj49ny5YtHD58+Lzfv3btWh5//HGuvPJKdu/eza5duwBoaWkhNTWVjIwMTpw4wa9+9SvWrVsHDLS/PntqaM2aNdx1111s2rQJ5xxPP/00P/rRj8Lyc4v/Gtt7eCZ04ndfbQsJcTFcu3AaG0vyuXRWlq74kfelIBghxcXFtLa2kpuby/Tp07njjju4/vrrWbRoESUlJcyfP/+83//5z3+eu+++m6KiIoqKilixYgUAS5YsYdmyZcyfP5/8/HxWr159+nvuvfderrnmGnJyctiyZcvp7cuXL+euu+5i5cqVAHzuc59j2bJlmgaKIP1BR9mBAJu3V/PbvSfo6Q+yKDeDv7uhmA1LcslIife7RBlH1IZahqT3dGw63NDOkztqeHJHDbXNXWSmxHPjslxuXZHPgpx0v8uTMUxtqEXGsc6efn61u5bN5dW8eugkMQZr503m659YwPqiKVptSz40BYHIGOSc463qJjaX1/DLncdo7e5jxqQU/tvV87hlRR7TM5L9LlEiSMQEgXNO10OPkPE2XRhJAm3dPPPmUTaXV/PuiTaS4mO4btF0Npbks7Jgkk78SlhERBAkJSXR0NBAVlaWwuBDcs7R0NBAUlKS36VEjb7+IC+9W8/m8mp+v6+OvqBj2YyJ/M+bF/GJxdNJS9KJXwmvsAaBmV0DfAeIBb7vnHvwrOdnAP8BTAzts8k5d8ENc/Ly8qipqaG+vn4EqpakpCTy8vL8LiPiHaxv44nyGp56o4b61m6yJyRw9+oCNpbkM3dqmt/lSRQJWxCYWSzwCPBRoAbYbmbPOef2Dtrta8Bm59x3zWwB8DxQcKF/Vnx8PIWFhSNQtUh4tXf38Z+7vBO/5YcbiY0xrrhoMreW5HPl/CnEa6lG8UE4PxGsBA445w4BmNnPgBuAwUHggFPXvGUAx8JYj4gvnHPsONzI5vJqfrmrlo6efmZNTmXTtfO5eVkuU9I1DSchzkFXE7Qcg5ZaaDkKraHfW2ph5b0w7+oR/2PDGQS5QPWgxzXAJWft8wDwGzP7UyAVuGqoFzKze4F7AWbMmDHihYqEQ11LF0+9cZQnyqs5FGgnNSGW6xfnsPHiPJbPyNT5rGgT7Ie2Omg9NsRAf2zgV98QbV9Sp0D69KGfGwF+nyz+FPBD59w/m9mlwI/MbKFzLjh4J+fco8Cj4N1Q5kOdIsPS2x/kD/vreKK8mi3v1NMfdFxckMl/XTebjy+aTqqWcoxMvV2hAT40qLcOMdC3HgfXf+b3xcR7A3xaDkxfDBddC2nTIT1n4NeEaRCXENbyw/mv8iiQP+hxXmjbYH8CXAPgnHvFzJKAbKAujHWJjLiKE61sLq/m6TePEmjrYUpaIveuncWtK/KYNXmC3+XJh9HTDk1HBqZnTg/0gwb7zpPv/b6ECQODeeFHQl9Ph/Tc0GCfCylZEOP/eaFwBsF2YK6ZFeIFwO3Ap8/a5wiwHvihmRUBSYAu/ZFxoaWrl1/u9E78vlXdRFyMcVXRVDZenMfauZOJ04nf8am/D47ugENb4NCLULMdgn1n7pM62RvMM3Ih/2LviP7sgT5p/LT8CFsQOOf6zOxLwAt4l4b+wDm3x8y+AZQ7554D/hz4npl9Fe/E8V1OdzPJGOac47XKk2zeXs3zu2vp6g0yb+oEvvbxIm5clkv2hMT3fxEZW5yDhgNwcIs3+FeVQXcLYJCzFC77U29xqlODfdo0iIusv+ewTliG7gl4/qxtfzvo673A6rO/T2SsqW3u5KkdNTyxo4bDDR2kJcZx8/I8NpbksyQvQyd+x5u2eqh8KTT4vwgtNd72iTNh4c0w6wooXAspk3wtc7TozJXIOXT39fO7vXVsLq+mtKKeoINLZ2Xxlavmck3xdJIT1Oxt3OjthMMve0f8B1+EE6HlbJMmegP+2j/3Bv9J0Xk/koJA5Cx7T63y9dZRmjp6mZ6RxJeumMMnV+QzI0urfI0LwSAc3zkw3XPkNejv9q7SmbEKrvw6zL4Cpi+FGAW6gkAEaO7o5bmd3ipfbx9tJiE2ho8WT+W2knxWz8kmVs3exr7GKm+a5+AWb9qns9HbPnUhrLzHO+KfeSkkpPpZ5ZikIJCo0dcfpKaxk8qGdg4H2qlq6KAy0E5VQzs1jZ30Bx1F09N54PoF3LA0l8zU8F67LR9SZyNUloame7ZAY6W3PS0HLroOZq3zLttMm+pnleOCgkAiSl9/kGNNXVQ2tFMVaKcy0M7hBm/Qrz7ZQV9w4KK01IRYCrJTWZibwQ1Lcri6eBoLczN8rF7Oq68bql/3jvoPbYFjb4ILetfrF6yBVZ/3Bv/seaCT9xdEQSDjTn/Qcayp8/QgXxnooCo08Fc3dtDbPzDYpyTEUpCVStH0NK5dOI2C7FQKs1MpyEole0KCrvYZy5yDur0DV/Yc3ga9HWCxkFcCa//Sm+fPXQGxatX9YSgIZEzqDzpqmzupCnScPrr3Bv12qk920tM/0IUkOT6WmVkpXDQtjY8tnEZBVgoFWd6APzktUYP9eOEcNByEqq3elE9VGbSHmgxkzYVld3rz/AWrIUmf3EaSgkB8FWjr5p3jraeP6E8d3R852UFP38BgnxgXQ0FWKnOmTOCqBVMpzEqlIHRkPzVdg/241VgVGvRLvd9bQw2I03K8o/3Ctd50T4bWxwgnBYGMOuccr1ee5IcvV/HCnuOcmrZPiIuhICuFwuxUrpw/hYKsVAqyvcdT05K0TGMkaD46MOhXboXmI9721MnePH/hGihYC1mzNc8/ihQEMmq6evt5bucxfritir21LWQkx3PP2lmsnTuZguxUpqdrsI84rSe8gb8qNPCfPORtT86Egsu99g2Fa2DyfA38PlIQSNgdb+7ix68e5vHXj3CyvYd5UyfwP29exI1Lc3V3bqRpb4DDZQNH/IF3vO2J6TBzNVz8Oe/If+rCMdF1UzwKAgkL5xxvHGnihy9X8au3a+l3jquKpnL3ZQVcOjtLc/qRorPJa91QudU76j+x29sen+rdvLX0094R/7QlEKvhZqzS34yMqO6+fp5/u5YfbqtiZ00zaUlx3HVZAX90aYHaM0SC7lY48qp3525lKRzf5V3LH5cE+ZfAlV/z5vhzl+uSznFEQSAjoq61i8dfO8KPXz1CoK2bWZNT+bsbirl5eZ5W5RrPejqg+tWBK3uOvuGtshUTD/krvWv5C9dA3sUR15o5muh/qHwou2qa+OG2Kv7vrmP09juuuGgyd68u5PI52TrxOx719XgLsZw64q/ZDsFe7yau3BVw+Ve8Of78SyBBn/AihYJALlhvf5Bf7z7OY9sqeeNIE6kJsdxxyUw+e1kBhdlq6DWuOAf1+wfu3q0qg952sBiYvsRr21C41uvYmZjmd7USJgoCGbaGtm5+tr2aH71ymOMtXRRkpXD/9Qv45Io80pI0HzxutB4f6NJ56EVoO+5tnzQbltzu3chVsAaSJ/pZpYwiBYG8r73HWvjhy5U889YxevqCrJmbzT/cvJB186Zo+mc86G4btCjLFqjf521PnuTdtTtrnTf4T5zhX43iKwWBDKmvP8jv9p3gsW1VvFZ5kuT4WDaW5PHZSwuYO1VTBGNaf5/XmfNUl87q1715/thE75LOU0f9UxfpWn4BFARylqaOHn6+vZr/88phjjZ1kpeZzN9cV8TGknwyUjT9MyY5592xe/AP3uBfWQrdzd5z0xbDpV/wmrXNWAXxyb6WKmOTgkAAePdEK49tq+LpN2vo6g1y6aws/vb6BVxVNFWrc41F7Q1Q+WJorv/FgZ49GfmwYEOoYdtHIDXbxyJlvFAQRLH+oGPL/joee7mSbQcaSIyL4aZludy1uoD509L9Lk8G6+2CI694Uz2HXoTaXYCDxAzvOv7V98HsK2HSLPXskQumIIhCgbZunn7jKD969TBHTnaQk5HEf79mPrdfnK/lGceKYBBOvD1o8fVXoa9r4EauK/7GO8mbs0ytG+RD07+gKNHXH+Sld+vZXF7N7/fV0Rd0XFyQyaZr53P1gqnExeqkoe86G2HfL725/sqXoKPB2z65CEr+OLT4+mWQOMHfOiXiKAgi3KH6Np7YUcNTO2qoa+0me0ICf3J5IbeW5DFniq7+8Z1z3lU9Ox6DPU97R/0TpsGcj3rz/LPWQdo0v6uUCKcgiEDt3X3859u1PFFezfaqRmJjjCsumsLGkjyumD+FeB39+6+rGXb+HHb8EOr2QEIaLL0Dlv+Rd0ev5vllFCkIIoRzjh2HG9lcXs0vd9XS0dPP7Mmp/NW187lpeS5T0pL8LlGcg6M7oPwx2P0U9HV6c/zX/wssvEVTPuIbBcE4V9fSxVNvHOWJ8moOBdpJTYhlw5Icbi3JZ/mMier7PxZ0tcDbm72j/+Nve736l9wGK+6GnKV+VyeiIBiPevuD/GF/HU+UV7PlnXr6g46VBZP4/LrZfHzxdFIS9Nc6Jhx9w5v7f/spr5HbtMXwif8Fi25VAzcZUzRijCMVJ1rZXF7N028eJdDWw5S0RP7L2ll8ckUesyZrWmFM6G6Dt5/wAqB2J8SnwMKbYcUfe4u16BOajEEKgjGupauXX+6sZXN5NW9VNxEfa1xVNJWNJfmsmZutyz7Hitqd3tz/209ATxtMKYbrHoLFGyEpw+/qRM5LQTAGOed4rfIkm7dX8/zuWrp6g8ybOoGvfbyIm5blkjVBK0GNCT3tsPsX3tH/0R3eco3FN0PJ3d6KXTr6l3FCQTCG1DZ38tSOGp7YUcPhhg7SEuO4ZXkeG0vyWZyXoRO/Y8WJPd7R/66fQ3cLTJ4P1/yjdwI4OdPv6kQumILAZ919/fx+Xx0/315NaUU9QQeXzc7iq1fN42PF00hOiPW7RAHo7fRu+Cp/DGpe91o6F9/oXfkzY5WO/mVcUxD4ZF9tC5vLq3nmzaM0dvSSk5HEl66cy60r8sifpLVgx4y6/d7Uz86fejeBZc2Fj/0DLPkUpEzyuzqREaEgGGU7Dp/kgef28vbRZhJiY7i62Dvxu3pOtto9jxW9XbD3WS8AjrwCsQlQtMGb+5+5Wkf/EnHCGgRmdg3wHSAW+L5z7sEh9tkIPAA4YKdz7tPhrMlvD/5qP8dbuvgfG4q5YWkOE1PU7XPMqH/Xu+lr5+NeA7hJs+GjfwdLP62+/hLRwhYEZhYLPAJ8FKgBtpvZc865vYP2mQv8FbDaOddoZlPCVc9Y0NrVyxtHmvgva2fx2csK/C5HwGv7cPD3UPYwVJV6bZ6LPuHN/Res0VKOEhXC+YlgJXDAOXcIwMx+BtwA7B20zz3AI865RgDnXF0Y6/Hdq4dO0h90rJk72e9SxDk48Dt48UE4Wg7pebD+flh2J0yI6OMRkfcIZxDkAtWDHtcAl5y1zzwAM9uGN330gHPu12e/kJndC9wLMGPGjLAUOxrKKupJjo9l+cyJfpcSvZyDit/AS//oXfufkQ+feNjr/BmnaTqJTn6fLI4D5gLrgDxgq5ktcs41Dd7JOfco8ChASUmJG+0iR0ppRYBVsyaRGKdLQkedc/DuC/DSg3DsTZg4A67/Diz5tAJAol44g+AokD/ocV5o22A1wGvOuV6g0szexQuG7WGsyxc1jR0cCrRzx6qZfpcSXZyDd37lfQKofQsmzoQN/593+WdsvN/ViYwJ4QyC7cBcMyvEC4DbgbOvCHoG+BTwmJll400VHQpjTb4pqwgAsGaurj4ZFc7B/v/0AuD4LsgshBsegcW3KQBEzhK2IHDO9ZnZl4AX8Ob/f+Cc22Nm3wDKnXPPhZ672sz2Av3AXzjnGsJVk59KDwSYmp7I3CnqEhpWwSDs/yW89E/e4u+TZsGN34VFG7XIu8g5hPV/hnPueeD5s7b97aCvHfBnoV8Rqz/o2HYgwPr5U9UvKFyCQdj3nBcAdXsgaw7c9O+w8JMKAJH3of8ho2DPsWaaOnpZO0/TQiMuGIS9z8DWb0HdXq8FxM3f85Z+jNFJeZHhUBCMgtLQ+YHVcxQEIybY7zWB2/otqN8P2RfBLf8bim9SAIhcIAXBKCirCFA0PZ1srSPw4QX7vTUAtn4LAu94LaA/+QNYcKMCQOQDUhCEWUdPH+WHT/LHqwv9LmV86++D3U95AdBQAZOL4JOPhQJAbSBEPgwFQZi9VnmS3n7H5bps9IPp7/OWf9z6LTh50FsC8tb/8LqBKgBERoSCIMxK3w2QEBfDxQXqXX9B+vu8FcBKH4KTh2DqItj4I5j/CQWAyAhTEIRZ2YF6LimcRFK85q+Hpb/XC4Ct34LGKpi2CG77CVx0nQJAJEwUBGF0oqWLd0+0ccvyPL9LGfv6e71VwLY+BE2HYfoSuP2ncNG1WghGJMwUBGFUerqthNpOn1Nfj7cQTOk/Q9MRyFkG1/4TzPuYAkBklCgIwqisop7sCQnMn5bmdylji3NQU+7dB7DnF9BaC7kr4Lp/hrkfVQCIjDIFQZgEg46yAwFWz8kmRmsRe4P/0R3e4L/3WWiu9tYCnr3e6wY65yoFgIhPhhUEZnYT8AfnXHPo8URgnXPumXAWN57tP95KoK0nuqeFnINjb4SO/J+F5iPeUpBz1sOVX/Pm/5My/K5SJOoN9xPB/c65p089cM41mdn9eG2kZQhlB+oBuDza2ko45y38svcZLwCaQoP/7Cvhir/2Bv9krdAmMpYMNwiGum5P00rnUVoRYN7UCUzLSPK7lPBzDmp3ho78n/au+omJg1lXwEc2wfzrIDnT7ypF5ByGO5iXm9m3gUdCj78I7AhPSeNfV28/r1ee5I5LIng1Mufg+NsDg39jZWjwXwdr/wLmfxxSdBOdyHgw3CD4U+DrwM8BB/wWLwxkCNurTtLdF4y81cicgxO7Q4P/M17LB4uFWR+BNX/m3fWrwV9k3BlWEDjn2oFNYa4lYpRVBIiPNS6ZFQGDonNen/9TR/4NB8BioHAtrL4P5l8PqVl+VykiH8Jwrxr6LXCrc64p9DgT+Jlz7mPhLG68Kq0IsGJmJikJ4/g0St2+gcE/8K43+BesgUu/6DV8S42wTzsiUWy4I1X2qRAAcM41mtmUMNU0rtW3drO3toW/+NhFfpdy4er2Dxr83/EG/5mr4ZL/6g3+E6L4UliRCDbcIAia2Qzn3BEAMyvAO1cgZ3n54Km2EuPkiLn+HW++f8/TUL8PMG/wX3mPN/inTfW7QhEJs+EGwd8AZWb2EmDAGuDesFU1jpVWBJiYEk9xzhi8Uaqvx5vmqdvnnfSt+K230DsGMy+D6x6CoushbZrflYrIKBruyeJfm1kJ3uD/Jt6NZJ3hLGw8cs5RWlHP6jnZxPrZViIY9K7lr9vnDfQn9npfN1RAsM/bJyYO8i72GovaRgIAAA6SSURBVLwVbYD06f7VKyK+Gu7J4s8BXwbygLeAVcArwJXhK238OVDXxomWbtaM5t3EbfXeVT11e+HEHm/Ar98PPW0D+0yc4a3sddG1MLUYpiyArDkQlzB6dYrImDXcqaEvAxcDrzrnrjCz+cA/hK+s8WlrqO10WJal7G7z5vNPH+GHfrXXD+yTkuUN8svuhClF3uA/ZT4kqvupiJzbcIOgyznXZWaYWaJzbr+ZjcPLYsKrrKKeWdmp5GWmfPAX6e+FhoNnTunU7fFW6zolPgUmz/d69k9Z4P2aWgypk9XBU0Qu2HCDoCbUcfQZ4Ldm1ggcDl9Z4093Xz+vHjrJrSXDXI3MOa8Vc92+0JROaNAPvAv9Pd4+FutN4eQsg6V3hAb8BTCxQMs2isiIGe7J4ptCXz5gZluADODXYatqHHqjqhHrbeejOb1eD57OJuhqGvR748DXzTXeoN/dMvAC6XneID9nvTelM3UBZM2F+ChoWicivrrgW1+dcy+Fo5AxwTno7Tj/IH6ObSs7G9mb1A//eY7Xtliv/XLSRO/yzMW3efP4U4u939WXX0R8Mo57IFyg2p1Qte08A3toe7D33K9hMd6AnTRxYFCfOAOSJ/KLPW10xqbxR1cs8Z5Lzjxzv8Q0zd+LyJgUPUFQuRV+8zXAICl9YJBOzoT0nIEBe6hB/PRgnj7k3Hxjew9/ue23fGX9PFgxd/R/NhGRDyF6gmDFXd5llYnpEBM7oi/98sEGnAvTZaMiImEWPUEQxmvpSyvqSUuKY0me5vlFZPzRNYgfktdWIsBls7OIi9XbKSLjj0auD6mqoYOjTZ1cPlctmkVkfFIQfEilFV6Lh7U6PyAi45SC4EMqrQiQPymZmVmpfpciIvKBKAg+hN7+IK8cbGCNpoVEZBwLaxCY2TVm9o6ZHTCzTefZ7xYzc6E1D8aNndVNtHX3jW7baRGRERa2IDCzWOAR4FpgAfApM1swxH5peG2uXwtXLeFSWhEgxuCy2QoCERm/wvmJYCVwwDl3yDnXA/wMuGGI/f4O+EegK4y1hEVpRT2L8yaSkRLvdykiIh9YOIMgF6ge9LgmtO00M1sO5DvnztWq7dR+95pZuZmV19fXn2/XUdPc2cvOmubxs0i9iMg5+Hay2MxigG8Df/5++zrnHnXOlTjnSiZPHhsnZl852EB/0OlEsYiMe+EMgqNA/qDHeaFtp6QBC4EXzawKbx3k58bLCeOyA/WkJsSybMZEv0sREflQwhkE24G5ZlZoZgnA7cBzp550zjU757KdcwXOuQLgVWCDc648jDWNmLKKAKtmZRGvthIiMs6FbRRzzvUBXwJeAPYBm51ze8zsG2a2IVx/7mioPtlBVUOHzg+ISEQIa/dR59zzwPNnbfvbc+y7Lpy1jKTSigCA+guJSETQvMYHUFpRT05GErMnq62EiIx/CoIL1B90vHywgcvnZmNaelJEIoCC4AK9fbSZ5s5eTQuJSMRQEFyg0nfrMYPL1V9IRCKEguAClR4IUJyTzqTUBL9LEREZEQqCC9DW3cebRxq5fI6mhUQkcigILsBrhxro7XdajUxEIoqC4AKUVgRIio9hRUGm36WIiIwYBcEFKK2o55LCLBLjYv0uRURkxCgIhqm2uZOD9e1qKyEiEUdBMEwDbSUUBCISWRQEw1RaEWByWiIXTU3zuxQRkRGlIBiGYNCx7UCANXPUVkJEIo+CYBj21rZwsr2HNfM0LSQikUdBMAynzg+sVlsJEYlACoJhKDtQz/xpaUxJS/K7FBGREacgeB+dPf1sr2zUZaMiErEUBO/j9aqT9PQH1XZaRCKWguB9lFXUkxAXw8qCSX6XIiISFgqC91FaEeDigkySE9RWQkQik4LgPOpauth/vFVtp0UkoikIzqPsgHfZqE4Ui0gkUxCcR1lFgKzUBBZMT/e7FBGRsFEQnINzjtIDAVbPySYmRm0lRCRyKQjO4Z0TrdS3dqvbqIhEPAXBOZRV6PyAiEQHBcE5bK0IMGfKBKZnJPtdiohIWCkIhtDV28/rlQ1criZzIhIFFARDeONwI129QU0LiUhUUBAMYWtFgPhYY9WsLL9LEREJOwXBEMoO1LNsRiapiXF+lyIiEnYKgrM0tHWz+2gLazUtJCJRQkFwlm0HGwDUdlpEooaC4CxlFfVkJMezKDfD71JEREaFgmAQ5xylFQFWz8kiVm0lRCRKKAgGOVjfTm1zl9pOi0hUCWsQmNk1ZvaOmR0ws01DPP9nZrbXzHaZ2e/NbGY463k/ZRX1gNpKiEh0CVsQmFks8AhwLbAA+JSZLThrtzeBEufcYuBJ4J/CVc9wlFYEKMhKIX9Sip9liIiMqnB+IlgJHHDOHXLO9QA/A24YvINzbotzriP08FUgL4z1nFdPX5BXDzWo26iIRJ1wBkEuUD3ocU1o27n8CfCroZ4ws3vNrNzMyuvr60ewxAFvHmmkvaefNbpsVESizJg4WWxmdwIlwLeGet4596hzrsQ5VzJ5cngG6rIDAWJjjEtnq62EiESXcPZQOArkD3qcF9p2BjO7Cvgb4CPOue4w1nNepRUBluZPJD0p3q8SRER8Ec5PBNuBuWZWaGYJwO3Ac4N3MLNlwL8DG5xzdWGs5byaO3rZVdOkttMiEpXCFgTOuT7gS8ALwD5gs3Nuj5l9w8w2hHb7FjABeMLM3jKz587xcmH18sEAQafLRkUkOoW1vaZz7nng+bO2/e2gr68K558/XFsrAqQlxrEkf6LfpYiIjLoxcbLYb2UH6lk1O4v4WL0dIhJ9on7kO9zQTvXJTrWdFpGoFfVBsLUiAKjttIhEr6gPgrKKenInJlOQpbYSIhKdojoI+vqDvHywgbXzsjFT22kRiU5RHQQ7a5pp7epT22kRiWpRHQRlFQHM4DK1lRCRKBbVQVBaUc/i3AwyUxP8LkVExDdRGwStXb28Wd2kttMiEvWiNghePXSS/qBT22kRiXpRGwSlFfWkJMSyfEam36WIiPgqaoOgrCLAJYWTSIiL2rdARASI0iCoaezgUKBd00IiIkRpEJSF2kqo7bSISJQGQemBANPSk5gzZYLfpYiI+C7qgqA/6Nh2IMDlc9VWQkQEojAI9hxrpqmjV9NCIiIhURcEpaHzA6u1PrGICBCVQVDPgunpZE9I9LsUEZExIaqCoKOnjx2HG1kzT58GREROiaogeO3QSXr7HWvUdlpE5LSoCoLSigCJcTGUFKithIjIKVEVBGUH6llZOImk+Fi/SxERGTOiJgiON3fx7ok2XTYqInKWqAmCsgPeZaNallJE5ExREwTpSXF8dMFU5k9L87sUEZExJc7vAkbL1cXTuLp4mt9liIiMOVHziUBERIamIBARiXIKAhGRKKcgEBGJcgoCEZEopyAQEYlyCgIRkSinIBARiXLmnPO7hgtiZvXA4Q/47dlAYATLGe/0fpxJ78cAvRdnioT3Y6ZzbsgeO+MuCD4MMyt3zpX4XcdYoffjTHo/Bui9OFOkvx+aGhIRiXIKAhGRKBdtQfCo3wWMMXo/zqT3Y4DeizNF9PsRVecIRETkvaLtE4GIiJxFQSAiEuWiJgjM7Boze8fMDpjZJr/r8YuZ5ZvZFjPba2Z7zOzLftc0FphZrJm9aWa/9LsWv5nZRDN70sz2m9k+M7vU75r8YmZfDf0/2W1mPzWzJL9rCoeoCAIziwUeAa4FFgCfMrMF/lblmz7gz51zC4BVwBej+L0Y7MvAPr+LGCO+A/zaOTcfWEKUvi9mlgvcB5Q45xYCscDt/lYVHlERBMBK4IBz7pBzrgf4GXCDzzX5wjlX65x7I/R1K95/8lx/q/KXmeUBHwe+73ctfjOzDGAt8L8BnHM9zrkmf6vyVRyQbGZxQApwzOd6wiJagiAXqB70uIYoH/wAzKwAWAa85m8lvnsY+Esg6HchY0AhUA88Fpoq+76ZpfpdlB+cc0eBh4AjQC3Q7Jz7jb9VhUe0BIGcxcwmAE8BX3HOtfhdj1/M7BNAnXNuh9+1jBFxwHLgu865ZUA7EJXn1MwsE2/moBDIAVLN7E5/qwqPaAmCo0D+oMd5oW1Ryczi8ULgJ865X/hdj89WAxvMrApvyvBKM/uxvyX5qgaocc6d+pT4JF4wRKOrgErnXL1zrhf4BXCZzzWFRbQEwXZgrpkVmlkC3gmf53yuyRdmZnjzv/ucc9/2ux6/Oef+yjmX55wrwPt38QfnXEQe9Q2Hc+44UG1mF4U2rQf2+liSn44Aq8wsJfT/Zj0ReuI8zu8CRoNzrs/MvgS8gHfm/wfOuT0+l+WX1cBngLfN7K3Qtr92zj3vY00ytvwp8JPQQdMh4G6f6/GFc+41M3sSeAPvars3idBWE2oxISIS5aJlakhERM5BQSAiEuUUBCIiUU5BICIS5RQEIiJRTkEgMorMbJ06nMpYoyAQEYlyCgKRIZjZnWb2upm9ZWb/HlqvoM3M/leoP/3vzWxyaN+lZvaqme0ys6dDPWowszlm9jsz22lmb5jZ7NDLTxjU7/8nobtWRXyjIBA5i5kVAbcBq51zS4F+4A4gFSh3zhUDLwH3h77l/wD/3Tm3GHh70PafAI8455bg9aipDW1fBnwFb22MWXh3e4v4JipaTIhcoPXACmB76GA9GajDa1P989A+PwZ+EerfP9E591Jo+38AT5hZGpDrnHsawDnXBRB6vdedczWhx28BBUBZ+H8skaEpCETey4D/cM791Rkbzb5+1n4ftD9L96Cv+9H/Q/GZpoZE3uv3wCfNbAqAmU0ys5l4/18+Gdrn00CZc64ZaDSzNaHtnwFeCq3+VmNmN4ZeI9HMUkb1pxAZJh2JiJzFObfXzL4G/MbMYoBe4It4i7SsDD1Xh3ceAeCzwL+FBvrB3To/A/y7mX0j9Bq3juKPITJs6j4qMkxm1uacm+B3HSIjTVNDIiJRTp8IRESinD4RiIhEOQWBiEiUUxCIiEQ5BYGISJRTEIiIRLn/B7Kr/SNMobLFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dd3hVVdbH8e9KJ9TQISGETiBAgNBBaSqiAooCir2g2Ms4lhlHxzKj7zjYGyoqCijSB7GhiCg19BIglAAJBEIPJaSt949zkSIJAXJzktz1eR6f4Z52V+48ye/us8/eW1QVY4wxvsvP7QKMMca4y4LAGGN8nAWBMcb4OAsCY4zxcRYExhjj4ywIjDHGx1kQGHMWIpIkIr3drsMYb7EgMMYYH2dBYIwxPs6CwJgCEpFgEXldRLZ7/ntdRII9+6qKyHQR2S8ie0Vkjoj4efY9ISIpIpIuIutEpJe7P4kxpwpwuwBjSpC/AR2BWECBqcDfgWeAx4BkoJrn2I6AikgT4H6gnapuF5EowL9oyzYmf9YiMKbghgLPq+ouVU0D/gnc5NmXBdQC6qpqlqrOUWcirxwgGGgmIoGqmqSqG12p3pg8WBAYU3C1gS0nvd7i2QbwH2AD8IOIbBKRJwFUdQPwMPAcsEtEvhSR2hhTjFgQGFNw24G6J72O9GxDVdNV9TFVrQ/0Ax493hegqmNVtavnXAVeKdqyjcmfBYExBTcO+LuIVBORqsA/gC8ARORKEWkoIgIcwLkllCsiTUSkp6dTOQM4CuS6VL8xZ2RBYEzBvQjEAyuAlcASzzaARsBM4BAwD3hXVWfh9A+8DOwGUoHqwFNFW7Yx+RNbmMYYY3ybtQiMMcbHWRAYY4yPsyAwxhgfZ0FgjDE+rsRNMVG1alWNiopyuwxjjClRFi9evFtVq51pX4kLgqioKOLj490uwxhjShQR2ZLXPq/dGhKROiIyS0TWiMhqEXnoDMeIiLwpIhtEZIWItPFWPcYYY87Mmy2CbOAxVV0iIuWBxSLyo6quOemYy3EG4jQCOgDvef7XGGNMEfFai0BVd6jqEs+/04EEIPy0w/oDo9UxH6gkIrW8VZMxxpg/K5I+As8c7K2BBaftCge2nfQ62bNtx2nnDwOGAURGRnqrTGOMC7KyskhOTiYjI8PtUkqFkJAQIiIiCAwMLPA5Xg8CESkHTAQeVtWD53MNVR0JjASIi4uzOTGMKUWSk5MpX748UVFROHP2mfOlquzZs4fk5GTq1atX4PO8Oo5ARAJxQmCMqk46wyEpQJ2TXkd4thljfERGRgZVqlSxECgEIkKVKlXOuXXlzaeGBPgYSFDVEXkcNg242fP0UEfggKruyONYY0wpZSFQeM7ns/Rmi6ALzjJ+PUVkmee/viJyj4jc4zlmBrAJZ2WnD4F7vVXMnkPH+Of/VnMsO8dbb2GMMSWSN58a+k1VRVVbqmqs578Zqvq+qr7vOUZV9T5VbaCqLVTVayPF5m/ayye/J3HfmKVk5di6IMYYx/79+3n33XfP+by+ffuyf/9+L1RU9HxmrqErWtbihf7NmZmwk4e/XEa2hYExhryDIDs7O9/zZsyYQaVKlbxVVpEqcVNMXIibOkVxLDuXF79JINBf+O+gWPz97N6kMb7sySefZOPGjcTGxhIYGEhISAhhYWGsXbuW9evXM2DAALZt20ZGRgYPPfQQw4YNA05Md3Po0CEuv/xyunbtyty5cwkPD2fq1KmUKVPG5Z+s4HwqCADu7FafzJxc/u+7dQQF+PHyNS3xszAwplj45/9Ws2b7eT1lnqdmtSvw7FXN89z/8ssvs2rVKpYtW8Yvv/zCFVdcwapVq/54/HLUqFFUrlyZo0eP0q5dOwYOHEiVKlVOuUZiYiLjxo3jww8/ZNCgQUycOJEbb7yxUH8Ob/K5IAC4t3tDjmXl8sZPiQQF+PFC/xh7asEYA0D79u1PeQb/zTffZPLkyQBs27aNxMTEPwVBvXr1iI2NBaBt27YkJSUVWb2FwSeDAODh3o04lp3L+7M3EuTvzzNXRlsYGOOy/L65F5WyZcv+8e9ffvmFmTNnMm/ePEJDQ+nevfsZn9EPDg7+49/+/v4cPXq0SGotLD4bBCLCE32acCw7h1G/byY40I+/XtbEwsAYH1O+fHnS09PPuO/AgQOEhYURGhrK2rVrmT9/fhFXVzR8NgjACYN/XNmMzOxc3vtlIyEB/jzUu5HbZRljilCVKlXo0qULMTExlClThho1avyxr0+fPrz//vtER0fTpEkTOnbs6GKl3iOqJWvqnri4OC3shWlyc5W/TlzBhMXJPNGnKcO7NyjU6xtj8paQkEB0dLTbZZQqZ/pMRWSxqsad6XifbhEc5+cnvDKwJZnZubzy3VqCA/y4vWvBJ2wyxpiSzILAw99PGDGoFVk5uTw/fQ1BAX7c2LGu22UZY4zX+czI4oII8PfjjSGt6dW0On+fsorx8dvOfpIxxpRwFgSnCQrw452hbejWqCpPTFzB1GU2K7YxpnSzIDiDkEB/Rt4UR4d6lXl0/HJmrLSZsY0xpZcFQR7KBPnz8S3tiK1TiQfHLWXmmp1ul2SMMV5hQZCPssEBfHJbO5rXrsC9Y5Ywe32a2yUZY1xWrlw5ALZv38611157xmO6d+/O2R5zf/311zly5Mgfr92c1tqC4CwqhAQy+vYONKxejmGj45m7cbfbJRljioHatWszYcKE8z7/9CBwc1prC4ICqBgayBd3dqBulVDu+DSeRUl73S7JGFNInnzySd55550/Xj/33HO8+OKL9OrVizZt2tCiRQumTp36p/OSkpKIiYkB4OjRowwZMoTo6GiuvvrqU+YaGj58OHFxcTRv3pxnn30WcCay2759Oz169KBHjx6AM6317t3OF80RI0YQExNDTEwMr7/++h/vFx0dzV133UXz5s259NJLC21OI6+NIxCRUcCVwC5VjTnD/orAF0Ckp45XVfUTb9VzoSqXDWLMnR0ZPHIet32yiC/u7EBsndKxKIUxxca3T0LqysK9Zs0WcPnLee4ePHgwDz/8MPfddx8A48eP5/vvv+fBBx+kQoUK7N69m44dO9KvX7885yJ77733CA0NJSEhgRUrVtCmTZs/9r300ktUrlyZnJwcevXqxYoVK3jwwQcZMWIEs2bNomrVqqdca/HixXzyyScsWLAAVaVDhw5cfPHFhIWFeW26a2+2CD4F+uSz/z5gjaq2AroD/xWRIC/Wc8GqlQ9m7J0dqVw2iJs/XsCqlANul2SMuUCtW7dm165dbN++neXLlxMWFkbNmjV5+umnadmyJb179yYlJYWdO/N+YOTXX3/94w9yy5Ytadmy5R/7xo8fT5s2bWjdujWrV69mzZo1+dbz22+/cfXVV1O2bFnKlSvHNddcw5w5cwDvTXfttRaBqv4qIlH5HQKUFydiywF7gfzXhisGalYMYexdHRj8wXxu+ngB44Z1pGnNCm6XZUzpkM83d2+67rrrmDBhAqmpqQwePJgxY8aQlpbG4sWLCQwMJCoq6ozTT5/N5s2befXVV1m0aBFhYWHceuut53Wd47w13bWbfQRvA9HAdmAl8JCqnnEhYREZJiLxIhKflnaeT+7k5kDK4vOt9RQRYaGMvauDMw3FRwvYsOtQoVzXGOOOwYMH8+WXXzJhwgSuu+46Dhw4QPXq1QkMDGTWrFls2bIl3/Mvuugixo4dC8CqVatYsWIFAAcPHqRs2bJUrFiRnTt38u233/5xTl7TX3fr1o0pU6Zw5MgRDh8+zOTJk+nWrVsh/rR/5mYQXAYsA2oDscDbInLGr9aqOlJV41Q1rlq1auf3bsvHwYc9YfJwOHzhT/7UrVKWsXd1BIQbPpxP0u7DF3xNY4w7mjdvTnp6OuHh4dSqVYuhQ4cSHx9PixYtGD16NE2bNs33/OHDh3Po0CGio6P5xz/+Qdu2bQFo1aoVrVu3pmnTptxwww106dLlj3OGDRtGnz59/ugsPq5NmzbceuuttG/fng4dOnDnnXfSunXrwv+hT+LVaag9t4am59FZ/A3wsqrO8bz+GXhSVRfmd83znoY68wj8+h+Y+xYElYXez0GbW8DvwrJwXWo6Q0bOIzQogK/u7khEWOgFXc8YX2PTUBe+c52G2s0WwVagF4CI1ACaAJu89m5BodD7WRj+O9SIgekPw6hLYceKC7psk5rl+fyODqRnZHHDhwvYcaBkLVFnjDFeCwIRGQfMA5qISLKI3CEi94jIPZ5DXgA6i8hK4CfgCVX1/mitak3g1ulw9QewdzOMvBi+ewqOnXmpuoKICa/I6Ds6sPdwJkM/XMCu9PPvDDLGmKLm2yuUHd0HM/8Jiz+F8jWhz8vQrD+c57rF8Ul7uXnUQsIrleHLYR2pUi747CcZ4+MSEhJo2rSprRdeSFSVtWvXlphbQ+4rEwZXvQ53zoSyVeHrW2DMtbD3/O5QxUVV5qNb4ti69wg3fbyQ/UcyC7lgY0qfkJAQ9uzZQ0n7UlocqSp79uwhJCTknM7z7RbByXKyYdGH8PNLkJMJ3R6Drg9DwLl/q5+9Po27Pounaa3yfHFnByqEBBZ+vcaUEllZWSQnJ1/Q8/XmhJCQECIiIggMPPXvTn4tAguC0x3cAd8/BasnQ5WGcMV/oX73c77MzDU7ueeLxbSMcPoPygXbqqDGGPfYraFzUaEWXPcp3DjJGYQ2uj9MuAPSU8/pMr2b1eCt61uzPPkAd3y6iKOZOd6p1xhjLpAFQV4a9oJ758PFT0LCNHi7HSwY6YRDAV3eohYjBrViYdJehn0eT0aWhYExpvixIMhPYAj0eMoJhPC28O3jzujklCUFvkT/2HD+b2BL5iTu5t4xS8jMPuMsGsYY4xoLgoKo0gBumgzXjnJuEX3YE755DI4WbDWh6+Lq8NLVMfy8dhcPjFtCVo6FgTGm+LAgKCgRiBkI9y+EDndD/CjndtGK8VCADvehHeryjyub8f3qnTzy1TILA2NMsWFBcK5CKsLlr8Bds6BiBEy6C0b3g7T1Zz319q71eOrypkxfsYPbP13EoWPFftZtY4wPsCA4X7VjnYFoV/wXti+H9zrDTy9AVv5zDd19cQNeGdiCuRv3MGTkPJuOwhjjOguCC+HnD+3uhAfiIeYamPMqvNMB1v+Q72mD20Xy4c1t2bjrMAPfm8umNFvPwBjjHguCwlCuOlwzEm75nzMSeex18NWNcCAlz1N6Nq3BuGEdOXwsh2vfn8fSrfuKsGBjjDnBgqAw1bsI7vkdej4DiT86nclz33amrziD2DqVmDi8M+WCA7j+w/n8lJD3mqjGGOMtFgSFLSAILvoL3LcAorrAD39zprreuuCMh9erWpaJwzvTqHp57hodz5cLtxZxwcYYX2dB4C1hUXDDeBj8hTPd9ahLYdoDcGTvnw6tVj6YL4d1pGujajw5aSWvz1xvMzEaY4qMBYE3iUD0VXDfQuj8ACwdA2+1hSWj/zRVRdngAD6+JY6BbSJ4fWYiT01aSbaNNTDGFAFvrlA2SkR2iciqfI7pLiLLRGS1iMz2Vi2uCy4Hl74I98yBqo2dlsG7HZ0ZTnNP/LEP9Pfj1etacl+PBny5aBt3f76YI5k21sAY413ebBF8CvTJa6eIVALeBfqpanPgOi/WUjzUaA63fQvXfQYIfH2r03+w/vs/RieLCI9f1pQX+jfn53W7uOHDBew5dMzVso0xpZvXgkBVfwX+fEP8hBuASaq61XP8Lm/VUqz4+UHzAXDvPGfd5GMHYewgGHUZbJ7zx2E3dYrivaFtSdhxkGvfn8fWPUdcLNoYU5q52UfQGAgTkV9EZLGI3JzXgSIyTETiRSQ+LS2tCEv0Ij9/aDUE7o+HK1+D/Vvhsyud9Q+SnYV3+sTUZMydHdh7OJNr3pvLqpQDLhdtjCmN3AyCAKAtcAVwGfCMiDQ+04GqOlJV41Q1rlq1akVZo/f5B0Lc7fDgUrj0JUhdCR/1gnHXQ+oq4qIqM3F4J4ID/Bj8wTx+XV9KgtAYU2y4GQTJwPeqelhVdwO/Aq1crMddgWWg8/3w0HLo8XdI+h3e7woTbqeh304m3duZOpVDuf3TRUxakux2tcaYUsTNIJgKdBWRABEJBToACS7WUzwEl4eLH4eHlkHXR2Ddt/BOe2rMeoyvr4+gXVRlHh2/nPd+2WhjDYwxhcKbj4+OA+YBTUQkWUTuEJF7ROQeAFVNAL4DVgALgY9UNc9HTX1OaGXo/azTQmg/DFaMp/zIDnxRewJDmwfzyndreW7aanJyLQyMMRdGStq3yri4OI2Pj3e7jKJ3IBlm/x8s/QL1D2Je1YHcm9SNTjGNeG1wLCGB/m5XaIwpxkRksarGnWmfjSwuKSpGQL834f5FSPRVdE4dw8Kyj9E44R3u/nAWB45kuV2hMaaEsiAoaao0gIEfwvC5BDXqziOBE3l956189cZf2J62x+3qjDElkAVBSVWjGQwZ4yyZGR7LsGOfEvhOW1JnvgnZmW5XZ4wpQSwISrrwNoQNm86Wfl+TIjWp+dszZLwWC0u/yHMdBGOMOZkFQSlRt82lVHvwZ54KfZbE9GCYep8zsd2qSadMbGeMMaezIChFwsNCeeL++3m+1tvcnfkw+zJyYMJt8MFFsO67Pya2M8aYk1kQlDKVQoP4/M6OSHQ/2u55nqn1nkUz02HcYPj4EthUemf7NsacHwuCUigk0J93hrbhxk71eCihCY9U+4isviPgQAqM7gefXQXbFrldpjGmmLAgKKX8/YR/9mvOE32aMmXFLm5Z3pyDdy+Ey/4NO9fAx73hi2udqa/tlpExPs2CoBQTEYZ3b8CIQa1YuHkvgz5ays7mtzvTVvR8BrYvdaa+/rCn06lsTxkZ45MsCHzANW0iGHVrO7btPcI1785lwwGFi/4Cj6yCK0ZAxn6nU/nttrDwQ8i0RXCM8SUWBD7iosbV+OruThzLzmXge/OIT9rrTH3d7g5ncZxBn0PZajDjL/Bac5j1Lzi82+2yjTFFwILAh8SEV2TS8M5ULhvE0I8W8N2qHc4OP39o1g/u+BFu+w7qdIDZrziBMP1R2LPR3cKNMV5ls4/6oL2HM7njs0Us3bqf+3s05JFLGuPvJ6celLYO5r4FK76CnCyIvgq6PAQRZ5y80BhTzOU3+6gFgY/KyMrhuWmr+XLRNro1qsobQ1pTuWzQnw9MT4UFH0D8x5BxACI7O4HQ6FLwswalMSWFBYHJ01eLtvLM1NVULRvEuze2JbZOpTMfeCwdloyGee/CwWSo2gQ6PwAtB0FAcNEWbYw5Z7YegcnT4HaRTLynMyLCoPfnMXbB1jMvgRlcHjrd5yyhec2H4B8E0+6H11vCb6/B0f1FX7wxplB4c6nKUSKyS0TyXX5SRNqJSLaIXOutWkz+WkRUZPoDXenYoApPT17J4xNWkJGVc+aD/QOdVsA9c+CmyVC9Kcx8zulY/v5vzkpqxpgSxZstgk+BPvkdICL+wCvAD16swxRAWNkgPrm1HQ/2asSExclc8+5ctu7JZzyBCDToCTdPhbt/hSaXw/z34I1WMGkYpNry08aUFF4LAlX9Fdh7lsMeACYCu7xVhyk4fz/h0UsaM+rWOJL3HeHKt+Ywa20B/q+p1QoGfuTcNmo/DBKmw/td4POrYdMvNoWFMcWca30EIhIOXA28V4Bjh4lIvIjEp6Wleb84H9ezaQ2mP9CNiLBQbvt0ESN+XE9ObgH+mFeKhD7/hkdXQ69/OK2C0f2dabBXTrApLIwpptzsLH4deEJVz7pqiqqOVNU4VY2rVq1aEZRmIquEMunezgxsE8GbPyVy+6eL2H+kgEtglgmDbo/Bwyvhqjch6yhMvAPebO3cPjp2yLvFG2POiVcfHxWRKGC6qsacYd9m4PgopqrAEWCYqk7J75r2+GjRUlXGLtzKP6etoXqFYN6/sS0x4RXP7SK5ubD+O5j7JmydByGVoN2d0OFuKFfdO4UbY05RLB8fVdV6qhqlqlHABODes4WAKXoiwtAOdRl/Tydyc5Vr3pvL+EXbzu0ifn7QtC/c/h3cMRPqdYM5/4XXYmDag84oZmOMa7z5+Og4YB7QRESSReQOEblHRO7x1nsa74mtU4npD3ajfVRl/jpxBU9NyucR0/zUaQeDv3Amuou9AZZ/Ce+0d/oS1s6A3PO4pjHmgtjIYnNOcnKVET+u451ZG2kRXpH3bmxDRFjo+V/w8G5Y/CnEj4KDKU6Hc7s7ofVNEFq50Oo2xtfZFBOm0P2wOpXHxi/H3194Y0hrLm58gZ34Odmw7htnPYSkORAQAi2uhfZ3Q62WhVO0MT7MgsB4xebdhxn+xWLW7Uznkd6Nub9HQ/xOn8X0fOxc7QTCiq8g6wjU6QgdhkF0P2dkszHmnFkQGK85kpnN05NWMmXZdno2rc5rg2KpGFpIf6yP7odlY5xQ2LcZytWEuNuh7a1QvkbhvIcxPsKCwHiVqvL5/C28MH0NtSqW4f0b29KsdoXCe4PcXNgwExaOhA0/gl8gNB/gjGKOaOdMd2GMyZcFgSkSi7fs494xi9l/JIt/Xd2CgW0jCv9N9myERR/B0i/g2EFneov2d0PMQAgMKfz3M6aUsCAwRSYt/RgPjFvC/E17ubFjJM9c2YzgAP/Cf6Njh5w+hIUjIW0tlKkMbW+BuDugUp3Cfz9jSjgLAlOksnNy+c8P6/hg9iZa1anEe0PbULtSGe+8marzlNGCD2DdDGdbk77ObaN6F9ltI2M8LAiMK75duYPHJ6wgKMCPt65vTZeGVb37hvu3OUtqLv4Mju6Fak2h/V3QcggEl/PuextTzFkQGNdsTDvEPZ8vZmPaIf5yWROGX9wA8fa39KwMWDURFn4AO5ZDcAWIHeqEQpUG3n1vY4opCwLjqsPHsnli4gqmr9jBJc1q8N9BragQUgTjAVQheZHTj7B6CuRmQcPeTudyw97OHEjG+AgLAuM6VeWT35P414wEIsLK8P5NbWlasxAfMT2b9J0nprI4lAph9ZwWQuxQKFOp6OowxiUWBKbYWJS0l3vHLOFQRjYvD2xB/9jwoi0gJwsSpjmD1LbOg8BQZw3m9sOgRvOircWYImRBYIqVXQczuH/sUhYm7eXWzlE83TeaoAAXbtPsWO4EwsqvITsDIjs5o5ab9YdALz3lZIxLLAhMsZOVk8vL367l49820zKiIiMGxdKwuktP9hzZ6wxQW/wp7N3oLJwTe4MTCtWauFOTMYXMgsAUW9+u3MFTk1dyNDOHpy5vys2dogpn4rrzcXxMQvwnkPA/p3M5svNJrQQbuWxKrgsOAhF5CPgESAc+AloDT6rqD4VZaEFYEJQ+uw5m8MTEFcxal0bXhlX5v2tbem8AWkEdSoPlYz2thE3OOsytrrdWgimxCiMIlqtqKxG5DLgbeAb4XFXbFG6pZ2dBUDqpKuMWbuPFb9bg7ye80D+G/rG1vT/m4Gxyc51WwuJPIGH6iVZC3G3OtNjWSjAlRGGsWXz8t7EvTgCsPmlbXm86SkR2iciqPPYPFZEVIrJSROaKSKsC1mJKIRHhhg6RzHiwG41rlOfhr5Zx/9il7Duc6W5hfn5Q/2K47lN4NAF6/xPSd8Cku2BEU/juaUhb726NxlyggrYIPgHCgXpAK8Af+EVV2+ZzzkXAIWC0qsacYX9nIEFV94nI5cBzqtrhbLVYi6D0y8lV3p+9kddnricsNIhXrm1JjybV3S7rhNxcSPrV6UtYOx1ys6FuF+e2kbUSTDFVGLeG/IBYYJOq7heRykCEqq44y3lRwPQzBcFpx4UBq1T1rA+VWxD4jtXbD/DIV8tYv/MQQztE8nTfaMoGB7hd1qkO7XIWz1n8KexL8vQlHH/iqLHLxRlzQmEEQRdgmaoeFpEbgTbAG6q65SznRVGwIPgL0FRV78xj/zBgGEBkZGTbLVvyfVtTimRk5TDix/V8OGcTkZVDGTEolrZ1w9wu689yc2HzbCcQTmkl3AbRV1krwbiuMIJgBc4toZbApzhPDg1S1YvPcl4UZwkCEekBvAt0VdU9Z6vFWgS+af6mPTw2fjk7DhxlePcGPNSrsTuD0AriT62Eys64hDa3WCvBuKYwgmCJqrYRkX8AKar68fFtZzkvinyCQERaApOBy1W1QD1uFgS+Kz0ji+f/t4avFyfTvHYFXhscS+Ma5d0uK29/tBI+gbXfeFoJXT3jEvpBQLDbFRofUhhBMBv4Drgd6AbsAparaouznBdFHkEgIpHAz8DNqjr3rEV4WBCYH1an8tSklaQfy+avlzXh9i713BuEVlCHdjmjl5d8dmoroe2tULWR29UZH1AYQVATuAFYpKpzPH/Eu6vq6HzOGQd0B6oCO4FngUAAVX1fRD4CBgLHb/hn51XkySwIDMDuQ8d4cuJKZibspEO9yvx3UCsiwkLdLuvscnNh8y/OE0frZjithKhunieOrrJWgvGaQpliQkRqAO08Lxeq6q5Cqu+cWBCY41SVrxcn889pqxERnr2qGde2jXB/EFpBpe+EZV84K6rt33KildD6Rqge7XZ1ppQpjBbBIOA/wC84A8m6AY+r6oRCrLNALAjM6bbtPcJjXy9n4ea9XNa8Bv+6ugVVypWgb9a5ubBpltO5fLyVUCvWCYWYa6FsFbcrNKVAoUwxAVxyvBUgItWAmapa5KOBLQjMmeTkKh//tolXv19PhTIBvHxNS3o3q+F2WefuUJozLfbysZC6EvwCofFlTig0vAQCgtyu0JRQhREEK0/uGPYMMDtrZ7E3WBCY/KxNPcgjXy0nYcdBBsfV4ZmrmlGuuA1CK6jUlbBsHKwcD4fTILQKtLjOmfyuVisoKbfATLFQGEHwH5wxBOM8mwYDK1T1iUKrsoAsCMzZHMvO4Y2Zibw/eyO1K5VhxKBY2ter7HZZ5y8nCzb85LQS1n0LOZlQvZkTCC0HQfmabldoSoDC6iweCHTxvJyjqpMLqb5zYkFgCio+aS+Pjl/Otn1HGNatPo9e2pjgAH+3y7owR/bC6klOSyElHsQPGvRybh016WsjmE2ebGEa47MOH8vmxW8SGLdwK01rlmfEoFia1a7gdlmFI209LB8HK76CgykQUhGaX+OEQkQ7u9DjS9cAABedSURBVHVkTnHeQSAi6cCZDhBAVbXIf6MsCMz5+HntTv46YSUHjmby6CVNGHZRffyL+yC0gsrNgc2/OqGwZhpkH4UqDaHVEGg5BCrVcbtCUwxYi8AYYO/hTP42eSXfrkolrm4YIwbFElmlBAxCOxcZB2HNVCcUtvwOCNTrBrFDnQFrQWXdrtC4xILAGA9VZfLSFJ6dupocVZ65shlD2tUpOYPQzsXezc5to+XjnGktgso5ay+3ut6ZGdWvmE7aZ7zCgsCY06TsP8rjXy9n7sY99GpanX8PbEH18qW0o1UVts6DZWNh9RTITIdKkc5to1ZDoEoDtys0RcCCwJgzyM1VPp2bxCvfrSU0yJ/n+jWnX6tisE6yN2UecWZCXT4WNs4CFOp0hNjrofnVToezKZUsCIzJR+LOdP7y9XKWJx+gW6OqvDgghrpVfOBe+oGUE7eOdq+HgBBoeqUTCvV7gF8Jf9TWnMKCwJizyMlVvpi/hf98v46snFwe6NmQYRc1KL6L3xQmVUhZ4rQSVk6AjP1QrqbTn9B8gNNisP6EEs+CwJgCSj2QwfPTVzNjZSoNq5fjpQExdKjvQ5O+ZR+D9d/BivGwYSZkZ1golBIWBMaco1lrd/HM1FUk7zvKdW0jeLpvNGFlfWzCt2PpsP57WD35tFDo5/QnWCiUKBYExpyHo5k5vPFTIh/N2USFMoE83TeagW3CS3dncl7yC4VmAyCyo/UpFHOuBIGIjAKuBHblsVSlAG8AfYEjwK2quuRs17UgMEVtbepBnp60kiVb99OxfmVeHNCChtXLuV2We46HwpopkPijJxRqOLePLBSKLbeC4CLgEDA6jyDoCzyAEwQdgDdUtcPZrmtBYNyQm6t8uWgbL3+bQEZWLvdcXJ97ezQkJNDH/+AdO+T0KZweCtGe20cWCsWGa7eGzrJ4/QfAL6o6zvN6Hc46yDvyu6YFgXFTWvoxXvpmDVOWbSeqSigvDmhB10ZV3S6reDh2CBI9t4/+FAoDILKThYKLimsQTAdeVtXfPK9/Ap5Q1T/9lReRYcAwgMjIyLZbtmw5/RBjitScxDSembKKpD1HGBBbm79f2YyqJWl5TG/7IxSmQOIPFgrFQIkPgpNZi8AUFxlZObw7awPvzd5ImUB/nrw8miHt6uBXWmY1LSwWCsVCcQ0CuzVkSoUNu9L52+RVLNi8l7Z1w/jX1S1oUrO822UVT6eEwo/OlNnlajgzoza/2kLBi4prEFwB3M+JzuI3VbX92a5pQWCKI1VlwuJk/jUjgfSMbO7sVp+HejWiTJD9UcvTsUNOC+GPPoWjULb6iUdS63a2UChEbj01NA7oDlQFdgLPAoEAqvq+5/HRt4E+OI+P3na220JgQWCKt72HM/n3jAS+XpxMRFgZXugfQ4+m1d0uq/g7HgprpsD6H06EQtMroPFlUO8iW0vhAtmAMmOK2PxNe/jb5JVsTDvMFS1q8Y+rmlGjQimd5rqwZR4+aZzCTMg6DP5BzhoKjS6FRpc4K7D54sC+C2BBYIwLjmXnMHL2Jt6atYFgfz/+clkTbuxYt/QskVkUso85aykk/uj8t3udsz0sChpe4gRDVFcIKmUrzXmBBYExLkrafZhnpq5iTuJuWkVU5KWrWxATbvP+n5d9W2CDJxQ2zXZuIQWEOGHQ6FJo2NsW2smDBYExLlNVpi3fzgvT17D3cCa3danHo5c0pmxwgNullVxZGc66zIk/OuGwZ4OzvXID5/ZRo0ugblcItFtyYEFgTLFx4EgWr3y/lrELtlKrYgjP9WvOZc1rul1W6bB3k9OnkPgDJM1xxisElHE6mo8HQ1iU21W6xoLAmGJm8ZZ9/G3yStampnNJsxr8s19zalcq43ZZpUfWUUj6zQmFxB9gX5KzvWpjT9/CJc7jqQG+MxrcgsCYYigrJ5dRv23m9ZmJiMCjlzTm1s5RBPjbHP+FShX2bPT0LfzgBEROJgSWhfoXO6HQ8BKoVMftSr3KgsCYYmzb3iM8O201P6/dRbNaFXj2qma+tSpaUcs8DJvneFoLP8KBrc72atHQqLfT6VynIwSUroWILAiMKeZUle9WpfL89DXsOJDBZc1r8OTl0dSraoOovEoVdq8/EQpb5kJuFgSV97QWPOMWKtR2u9ILZkFgTAlxNDOHj3/bxLu/bCQzO5ebOtXlwZ6NfG+ZTLccS4fNv54IhoMpzvZqTZ15kOp2gbqdoGKEu3WeBwsCY0qYXekZvPZjIl8t2kq54AAe7NWImzrVJTjA5t4pMqqwK+FEv8LW+ZCZ7uyrFOmEwvFwqNKg2I90tiAwpoRal5rOv2YkMHt9GpGVQ3ny8qZcHlPTN9dNdltuDqSudEY6b/kdtsyDI7udfWWrOy2F4+FQo3mxmzDPgsCYEu7X9Wm89E0C63amE1c3jL9dEU3ryDC3y/JtqrA7EbbOdfoWtsyFA9ucfcEVIbKD84hq3S5QK9b1zmcLAmNKgZxcZXz8Nv77w3p2HzrGVa1q89fLmlCnss2zU2zs3+q0FI6Hw+71zvaAMhAR5wmGzhDRrshnU7UgMKYUOXQsmw9mb+TDOZvIVbitSxT39WhIhZBAt0szpzuU5rmVNNcJh9SVoLngF+C0Eo4HQ2RHKOPdFp4FgTGl0I4DR/nP9+uYtCSFymWDeLh3I65vH0mgDUgrvjIOwraFnj6GubB9iTO4DYHqzU4EQ93OUL5wpx6xIDCmFFuVcoAXv1nD/E17aVCtLE9dHk2v6OrWoVwSZB2FlMXO7aQtvzshkXXY2Ve5PkSeFAxhURf0ZJIFgTGlnKoyM2EX/56RwKbdh+lUvwp/uyLaprsuaXKyIXW5Jxg8t5OO7nP2la8FXR6Gjvec16XdXLO4D/AG4A98pKovn7Y/EvgMqOQ55klVnZHfNS0IjMlbVk4uYxds5fWZ69l/NItrWkfw+GVNqFnRpmIukXJzncV4jj+u2uhSaDX4vC7l1prF/sB64BIgGVgEXK+qa046ZiSwVFXfE5FmwAxVjcrvuhYExpzdgaNZvDtrA5/8noSfHwzrVp+7L25g6x/4sPyCwJu9Su2BDaq6SVUzgS+B/qcdo0AFz78rAtu9WI8xPqNimUCe6hvNT49dTO/oGrz58wa6v/oLXy3aSk5uybodbLzPm0EQDmw76XWyZ9vJngNuFJFkYAbwwJkuJCLDRCReROLT0tK8UasxpVKdyqG8fUMbJt3bmTphZXhi4kqueHMOcxLt98ic4PZzZtcDn6pqBNAX+FxE/lSTqo5U1ThVjatWrVqRF2lMSdcmMoyJwzvzzg1tOJyZzU0fL+TWTxayfme626WZYsCbQZACnLzSQ4Rn28nuAMYDqOo8IASo6sWajPFZIsIVLWsx89GL+VvfaBZv2Uef13/l6ckrSUs/5nZ5xkXeDIJFQCMRqSciQcAQYNppx2wFegGISDROEFib1RgvCg7w566L6vPr4z24uVMU4xdto8erv/DOrA1kZOW4XZ5xgdeCQFWzgfuB74EEYLyqrhaR50Wkn+ewx4C7RGQ5MA64VUvawAZjSqiwskE81685PzxyEZ0bVOE/36+j56u/MHlpMrnWoexTbECZMQaA+Zv28OI3a1iVcpAW4RV5uHcjeja1EcqlhVuPjxpjSpCO9asw7b6ujBjUiv1HM7njs3iuevs3vl+dai2EUs5aBMaYP8nKyWXK0hTembWBpD1HaFqzPA/2akSf5jXx87MWQklkcw0ZY85Ldk4u/1uxnbd+3sCmtMM0rlGO+3s24ooWtfC3QChRLAiMMRckJ1f5ZuUO3vopkcRdh6hfrSwP9GzIVS1rE2DTXpcIFgTGmEKRm6t8tzqVN39KZG1qOlFVQrmvR0MGtA63dRCKOQsCY0yhys1VfkzYyZs/JbJ6+0HqVC7Dfd0bck2bCIICLBCKIwsCY4xXqCo/r93Fmz8lsjz5AOGVyjC8ewOui4sgOMDf7fLMSSwIjDFeparMXp/GGz8lsnTrfmpWCGF49wYMbleHkEALhOLAgsAYUyRUld837OGNn9azKGkf1csHc/fFDbihfSRlgiwQ3GRBYIwpUqrK/E17efOnROZt2kPVckEMu6g+QzvUtcVxXGJBYIxxzcLNe3nr50TmJO6mctkg7uxWj5s7RVHOAqFIWRAYY1y3eMs+3vo5kV/WpVGxTCB3dK3HLZ2jqFgm0O3SfIIFgTGm2Fi+bT9v/ZzIzIRdlA8J4LYu9bijSz0qhlogeJMFgTGm2FmVcoC3fk7k+9U7KRccwC2d63JH1/pULhvkdmmlkgWBMabYSthxkLd/3sCMVTsoE+jPTZ3qcle3+lQtF+x2aaWKBYExpthbvzOdt3/ewP9WbCc4wI8bO9Rl2EX1qV4hxO3SSgXX1iMQkT4isk5ENojIk3kcM0hE1ojIahEZ6816jDHFV+Ma5Xnz+tb8+MjF9I2pxajfN9PllZ95+MulLN26j5L2pbUk8VqLQET8gfXAJUAyzhrG16vqmpOOaYSzeH1PVd0nItVVdVd+17UWgTG+IWn3YT6dm8SExckcOpZNy4iK3Nwpiitb1rLRyufBlVtDItIJeE5VL/O8fgpAVf990jH/B6xX1Y8Kel0LAmN8y6Fj2Uxeksxn87awYdchwkIDGdI+kqEdIokIC3W7vBIjvyDw5oiOcGDbSa+TgQ6nHdMYQER+B/xxguM7L9ZkjClhygUHcFOnKG7sWJd5G/fw2bwkPpi9kQ9mb6R3dA1u7RxFpwZVbG3lC+D20L4AoBHQHYgAfhWRFqq6/+SDRGQYMAwgMjKyqGs0xhQDIkLnhlXp3LAqyfuOMGbBVr5cuJUf1uykYfVy3NKpLle3ibARy+fBm53FKUCdk15HeLadLBmYpqpZqroZp0+h0ekXUtWRqhqnqnHVqlXzWsHGmJIhIiyUJ/o0Zd5TvXj1ulaUCfTnmamr6fivn3hu2mo2ph1yu8QSxZt9BAE4f9h74QTAIuAGVV190jF9cDqQbxGRqsBSIFZV9+R1XesjMMacTlVZtm0/o+dtYfqK7WTlKN0aVeXmTlH0bFrd1lfGxXEEItIXeB3n/v8oVX1JRJ4H4lV1mjg39f4L9AFygJdU9cv8rmlBYIzJT1r6Mb5atJUv5m8l9WAG4ZXKcFOnugyOq0OYD49atgFlxhifk5WTy8w1O/lsXhLzN+0lOMCP/rG1ublTFDHhFd0ur8hZEBhjfNra1IOMnreFyUtSOJqVQ9u6YdzcqS6Xx9TymTWWLQiMMQY4cDSLCYuT+XxeEkl7jlC1XDA3dHDGJNQo5VNZWBAYY8xJcnOVXxPTGD1vC7PW7cJfhD4xNbmlcxRxdcNK5ZgEtwaUGWNMseTnJ3RvUp3uTaqzZc9hvpi/ha8WbWP6ih1E16rALZ3q0j823GfWWbYWgTHGAEczc5iyLIXP5iaxNjWdCiEBDG5Xh5s6RhFZpeRPZWG3howxpoBUlUVJ+/hsXhLfrUolV5WuDasyIDacy2JqltiRyxYExhhzHlIPZDB24VYmLUkmed9RQgL9uKRZTQbE1uaixtUI9C85TxxZEBhjzAVQVRZv2ceUZSl8s2IH+45kERYayBUtazEgNpy2JaCD2YLAGGMKSWZ2LnMS05iybDs/rkklIyuXiLAy9I+tzYDYcBrVKO92iWdkQWCMMV5w6Fg2P6xOZcqy7fyWmEauQrNaFRjQujb9WoVTs2LxGZtgQWCMMV62Kz2D6ct3MHVZCsuTDyACHetVYUDr2vSJqUXFMoGu1mdBYIwxRWjz7sNMWZrC1GUpJO05QlCAHz2bVGdA63B6NK1GcEDRj0+wIDDGGBeoKsuTDzBlaQrTV2xn96FMKoQE0LdFLfrHhtOhXmX8imiKbAsCY4xxWXZOLr9v3MPUpSl8vzqVw5k51KoYQr9WtekfG050rfJeffLIgsAYY4qRo5k5/Jiwk6lLU5i9Po3sXKVxjXL0jw2nf2xtIsIKfySzBYExxhRTew9n8s3KHUxdmkL8ln0AtIsKo39sOFe0qFVoi+lYEBhjTAmwbe8Rpi5LYcqy7WzYdYhAf+HixtXoHxtO7+gaFzQJnptLVfYB3sBZqvIjVX05j+MGAhOAdqqa7195CwJjTGmnqqzefpCpy1KYtnw7Ow8eo2yQP49c0pg7u9U/r2u6Mg21iPgD7wCXAMnAIhGZpqprTjuuPPAQsMBbtRhjTEkiIsSEVyQmvCJPXh7Ngk17mLIsxWsD1Lw5jV57YIOqbgIQkS+B/sCa0457AXgFeNyLtRhjTInk7yd0bliVzg2reu09vDl1Xjiw7aTXyZ5tfxCRNkAdVf0mvwuJyDARiReR+LS0tMKv1BhjfJhrc6iKiB8wAnjsbMeq6khVjVPVuGrVqnm/OGOM8SHeDIIUoM5JryM8244rD8QAv4hIEtARmCYiZ+zMMMYY4x3eDIJFQCMRqSciQcAQYNrxnap6QFWrqmqUqkYB84F+Z3tqyBhjTOHyWhCoajZwP/A9kACMV9XVIvK8iPTz1vsaY4w5N15dfFNVZwAzTtv2jzyO7e7NWowxxpxZyVlw0xhjjFdYEBhjjI8rcXMNiUgasOU8T68K7C7Ecko6+zxOZZ/HCfZZnKo0fB51VfWMz9+XuCC4ECISn9dcG77IPo9T2edxgn0Wpyrtn4fdGjLGGB9nQWCMMT7O14JgpNsFFDP2eZzKPo8T7LM4Van+PHyqj8AYY8yf+VqLwBhjzGksCIwxxsf5TBCISB8RWSciG0TkSbfrcZOI1BGRWSKyRkRWi8hDbtfkNhHxF5GlIjLd7VrcJiKVRGSCiKwVkQQR6eR2TW4RkUc8vyOrRGSciHhniTCX+UQQnLRs5uVAM+B6EWnmblWuygYeU9VmONN/3+fjnwc4y6UmuF1EMfEG8J2qNgVa4aOfi4iEAw8Ccaoag7P2+hB3q/IOnwgCTlo2U1UzgePLZvokVd2hqks8/07H+UUPz/+s0ktEIoArgI/crsVtIlIRuAj4GEBVM1V1v7tVuSoAKCMiAUAosN3lerzCV4LgrMtm+ioRiQJaAwvcrcRVrwN/BXLdLqQYqAekAZ94bpV9JCJl3S7KDaqaArwKbAV2AAdU9Qd3q/IOXwkCcwYiUg6YCDysqgfdrscNInIlsEtVF7tdSzERALQB3lPV1sBhwCf71EQkDOfOQT2gNlBWRG50tyrv8JUgONuymT5HRAJxQmCMqk5yux4XdQH6eZZL/RLoKSJfuFuSq5KBZFU93kKcgBMMvqg3sFlV01Q1C5gEdHa5Jq/wlSDId9lMXyMignMPOEFVR7hdj5tU9SlVjfAslzoE+FlVS+W3voJQ1VRgm4g08WzqBaxxsSQ3bQU6ikio53emF6W049yrK5QVF6qaLSLHl830B0ap6mqXy3JTF+AmYKWILPNse9qzopwxDwBjPF+aNgG3uVyPK1R1gYhMAJbgPGm3lFI61YRNMWGMMT7OV24NGWOMyYMFgTHG+DgLAmOM8XEWBMYY4+MsCIwxxsdZEBhThESku81waoobCwJjjPFxFgTGnIGI3CgiC0VkmYh84Fmv4JCIvOaZn/4nEanmOTZWROaLyAoRmeyZowYRaSgiM0VkuYgsEZEGnsuXO2m+/zGeUavGuMaCwJjTiEg0MBjooqqxQA4wFCgLxKtqc2A28KznlNHAE6raElh50vYxwDuq2gpnjpodnu2tgYdx1saojzPS2xjX+MQUE8aco15AW2CR58t6GWAXzjTVX3mO+QKY5Jm/v5KqzvZs/wz4WkTKA+GqOhlAVTMAPNdbqKrJntfLgCjgN+//WMacmQWBMX8mwGeq+tQpG0WeOe24852f5dhJ/87Bfg+Ny+zWkDF/9hNwrYhUBxCRyiJSF+f35VrPMTcAv6nqAWCfiHTzbL8JmO1Z+S1ZRAZ4rhEsIqFF+lMYU0D2TcSY06jqGhH5O/CDiPgBWcB9OIu0tPfs24XTjwBwC/C+5w/9ybN13gR8ICLPe65xXRH+GMYUmM0+akwBicghVS3ndh3GFDa7NWSMMT7OWgTGGOPjrEVgjDE+zoLAGGN8nAWBMcb4OAsCY4zxcRYExhjj4/4fP0TJcynmbwQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_history(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the trained model on the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Test Set Metrics:\n",
      "\tloss: 0.7819\n",
      "\tacc: 0.8109\n"
     ]
    }
   ],
   "source": [
    "test_gen = generator.flow(test_data.index, test_targets)\n",
    "\n",
    "test_metrics = model.evaluate_generator(test_gen)\n",
    "print(\"\\nTest Set Metrics:\")\n",
    "for name, val in zip(model.metrics_names, test_metrics):\n",
    "    print(\"\\t{}: {:0.4f}\".format(name, val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check serialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save model\n",
    "model_json = model.to_json()\n",
    "model_weights = model.get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load model from json & set all weights\n",
    "model2 = models.model_from_json(\n",
    "    model_json, custom_objects={\"GraphAttention\": GraphAttention}\n",
    ")\n",
    "model2.set_weights(model_weights)\n",
    "model2_weights = model2.get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "pred2 = model2.predict_generator(test_gen)\n",
    "pred1 = model.predict_generator(test_gen)\n",
    "print(np.allclose(pred1, pred2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Node and link importance via saliency maps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define the importances of node features, nodes, and links in the target node's neighbourhood (ego-net), and evaluate them using our library.\n",
    "\n",
    "Node feature importance: given a target node $t$ and the model's prediction of $t$'s class, for each node $v$ in its ego-net, feature importance of feature $f$ for node $v$ is defined as the change in the target node's predicted score $s(c)$ for the winning class $c$ if feature $f$ of node $v$ is perturbed.\n",
    "\n",
    "The overall node importance for node $v$ is defined here as the sum of all feature importances for node $v$, i.e., it is the amount by which the target node's predicted score $s(c)$ would change if we set all features of node $v$ to zeros.\n",
    "\n",
    "Link importance for link $e=(u, v)$ is defined as the change in target node $t$'s predicted score $s(c)$ if the link $e$ is removed from the graph. Links with high importance (positive or negative) affect the target node prediction more than links with low importance.\n",
    "\n",
    "Node and link importances can be used to assess the role of neighbour nodes and links in model's predictions for the node(s) of interest (the target nodes). For datasets like CORA-ML, the features and edges are binary, vanilla gradients may not perform well so we use integrated gradients to compute them (https://arxiv.org/pdf/1703.01365.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from stellargraph.utils.saliency_maps import IntegratedGradientsGAT\n",
    "from stellargraph.utils.saliency_maps import GradientSaliencyGAT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select the target node whose prediction is to be interpreted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "graph_nodes = list(G.nodes())\n",
    "all_gen = generator.flow(graph_nodes)\n",
    "target_idx = 7\n",
    "target_nid = graph_nodes[target_idx]\n",
    "target_gen = generator.flow([target_nid])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Node id of the target node:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true = all_targets[target_idx]  # true class of the target node"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract adjacency matrix and feature matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "target node id: 1109199, \n",
      "true label: [0. 0. 1. 0. 0. 0. 0.], \n",
      "predicted label: [0.18 0.33 0.15 0.09 0.18 0.03 0.04]\n"
     ]
    }
   ],
   "source": [
    "y_pred = model.predict_generator(target_gen).squeeze()\n",
    "class_of_interest = np.argmax(y_pred)\n",
    "print(\n",
    "    \"target node id: {}, \\ntrue label: {}, \\npredicted label: {}\".format(\n",
    "        target_nid, y_true, y_pred.round(2)\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the node feature importance by using integrated gradients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "int_grad_saliency = IntegratedGradientsGAT(model, train_gen, generator.node_list)\n",
    "saliency = GradientSaliencyGAT(model, train_gen)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the ego network of the target node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "G_ego = nx.ego_graph(Gnx, target_nid, radius=len(gat.activations))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute the link importance by integrated gradients."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "integrated_link_mask.shape = (2708, 2708)\n"
     ]
    }
   ],
   "source": [
    "integrate_link_importance = int_grad_saliency.get_link_importance(\n",
    "    target_nid, class_of_interest, steps=25\n",
    ")\n",
    "print(\"integrated_link_mask.shape = {}\".format(integrate_link_importance.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "integrated_node_importance [0. 0. 0. ... 0. 0. 0.]\n",
      "integrated self-importance of target node 1109199: 0.0\n",
      "\n",
      "Ego net of target node 1109199 has 202 nodes\n",
      "Number of non-zero elements in integrated_node_importance: 215\n"
     ]
    }
   ],
   "source": [
    "integrated_node_importance = int_grad_saliency.get_node_importance(\n",
    "    target_nid, class_of_interest, steps=25\n",
    ")\n",
    "print(\"\\nintegrated_node_importance\", integrated_node_importance.round(2))\n",
    "print(\n",
    "    \"integrated self-importance of target node {}: {}\".format(\n",
    "        target_nid, integrated_node_importance[target_idx].round(2)\n",
    "    )\n",
    ")\n",
    "print(\n",
    "    \"\\nEgo net of target node {} has {} nodes\".format(target_nid, G_ego.number_of_nodes())\n",
    ")\n",
    "print(\n",
    "    \"Number of non-zero elements in integrated_node_importance: {}\".format(\n",
    "        np.count_nonzero(integrated_node_importance)\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the ranks of the edge importance values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "sorted_indices = np.argsort(integrate_link_importance.flatten().reshape(-1))\n",
    "sorted_indices = np.array(sorted_indices)\n",
    "integrated_link_importance_rank = [(int(k / N), k % N) for k in sorted_indices[::-1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 10 most important links by integrated gradients are [(1544, 163), (1206, 163), (1544, 1206), (566, 733), (566, 466), (566, 186), (566, 1816), (566, 294), (566, 483), (566, 1447)]\n"
     ]
    }
   ],
   "source": [
    "topk = 10\n",
    "print(\n",
    "    \"Top {} most important links by integrated gradients are {}\".format(\n",
    "        topk, integrated_link_importance_rank[:topk]\n",
    "    )\n",
    ")\n",
    "# print('Top {} most important links by integrated gradients (for potential edges) are {}'.format(topk, integrated_link_importance_rank_add[-topk:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following, we plot the link and node importance (computed by integrated gradients) of the nodes within the ego graph of the target node.\n",
    "\n",
    "For nodes, the shape of the node indicates the positive/negative importance the node has. 'round' nodes have positive importance while 'diamond' nodes have negative importance. The size of the node indicates the value of the importance, e.g., a large diamond node has higher negative importance.\n",
    "\n",
    "For links, the color of the link indicates the positive/negative importance the link has. 'red' links have positive importance while 'blue' links have negative importance. The width of the link indicates the value of the importance, e.g., a thicker blue link has higher negative importance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "nx.set_node_attributes(\n",
    "    G_ego, values={x[0]: {\"subject\": x[1]} for x in node_data[\"subject\"].items()}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAv4AAAIuCAYAAADHWwyWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAXBUlEQVR4nO3dX6zfd33f8dfn+DgJiR27mU3+AFpILP+JT/7NBqljFJKmhDYVQ4KiVNTKRhFcdQUmRiOqqCUa2sCw9mqTlTqt1BA1KphNSMsGbS0SaEE2ScZJnGlJygrBTuyGOHZiktjnsws7xUtzjg/hnPM9P78fD+koPj5vf39vOzdPf/w557TeewAAgNPb2NALAAAA80/4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFDA+9AIAADAq1rTWn1ug19qb/I/e+zvn6nnCHwAAZum5JB9eoNf6vWTVXD7PVR8AAChA+AMAQAHCHwAACnDHHwBgHu3evfu14+PjtyWZiEPXqSSTR48e/eCmTZueHHqZaoQ/AMA8Gh8fv+2CCy7YsHr16h+NjY31ofcZ0tTUVNu/f/9l+/btuy3Ju4bep5rqf+sEAJhvE6tXr36mevQnydjYWF+9evXBHP/XDxaY8AcAmF9jov8nTvxZaNABuOoDAHAa27dv35K3v/3t65LkwIEDS8fGxvp55513NEnuv//+PWedddac/6Xk3nvvPXvfvn3j733ve5+Z62fz6gl/AIDT2AUXXHDs4YcffihJPvaxj120bNmyY5/61KeemO2vP3r0aMbHf7pk/Pa3v3325OTka4T/4uKfWQAAirr22mvXbNy4ccOaNWs2fv7zn1+VJC+++GKWL19+1Qc+8IE3rF279rKdO3eec8cdd6y4+OKLJzZu3LjhpptuesN11113aZIcPHhw7D3vec/Fl19++YYNGzZc9oUvfGHF4cOH22c/+9kLd+zYcd769esvu/32239u2N8lL3HiDwBQ1J133vm3559//rFDhw6NXXXVVRu2bNnyo5UrVx47fPjwkre97W2Htm/f/v1Dhw6NrVmzZuKee+55eM2aNS/ccMMNl7z06z/xiU9cdP311x/84he/+L39+/cvedOb3rTh3e9+94Mf//jH905OTr5m+/bt3x/y98f/z4k/AEBRn/70p89ft27dZZs3b17/xBNPnLFnz54zk2Tp0qV9y5YtTyfJfffdd9Yll1zy47Vr174wNjaWG2+88amXfv3OnTvP3bp164Xr16+/7K1vfeu6559/vj3yyCNnDPX7YWZO/AEACvryl7+8/Jvf/Oby3bt371m2bFnftGnTuiNHjowlyZlnnjk1Nnbq8+Hee3bs2PHoxo0bnz/557/2ta8tn6e1+Rk48QcAKOjpp59esnLlyqPLli3ru3btOuu73/3uOa80d/XVV//4scceO+uRRx5ZOjU1lbvuuuu8lz52zTXXPPO5z33utS+9/41vfOM1SbJ8+fJjhw8f1pmLjP8hAAAFve997zt45MiRsUsvvXTjzTff/Lorrrji2VeaW758+dTWrVv/7rrrrlt3+eWXb1ixYsWxc88991iSfOYzn/nhc889N7Z27drL1qxZs/GWW265KEluuOGGQw899NDZGzZs8Mm9i0jr3feTAACYLw888MD3rrzyygND7/GzOHjw4NiKFSumpqam8v73v/+fTkxMHPnkJz/55Kt93gMPPLDqyiuvvHgOV1wwF7XWP7xAr/V7ye7e++a5ep4TfwAAZrR169bV69evv2zNmjUbjxw5MvbRj350/9A78dPzyb0AAMzo1ltvfeLWW2+d9Tf9YnFy4g8AAAUIfwCA+TU1NTXVhl5isTjxZzE19B4VCX8AgPk1uX///hXi/3j079+/f0WSyaF3qcgdfwCAeXT06NEP7tu377Z9+/ZNxKHrVJLJo0ePfnDoRSoS/gAA82jTpk1PJnnX0HtA9b91AgBACcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AAAwglpr61pr95/09kxr7SPTzY8v5HIAAMDc6L3/7yRXJUlrbUmSx5PsmG7eiT8AAIy+X0zyaO/9/0434MQfgDnXJrI5yYeTXJLkqSR/muQrfTLHBl0M4PR1Y5I7ZxoQ/gDMmTaRFUn+a5I3JTkzyZITH7o+yaE2kXf0yTw41H4AI2ZVa23XSe9v671ve/lQa+2MJO9KcvNMDxP+AMyJNpElSb6W5PIcj/6TLU+yLMk9bSJX9sl8f6H3AxhBB3rvm2cx98tJvtN7f2KmIXf8AZgrNyRZn38c/S9pOR7/v7NgGwHU8Os5xTWfRPgDMHc+luNhP5OlSW5qEzljAfYBOO211s5J8ktJvnSqWeEPwFxZN8u5luS187kIQBW992d77/+k937wVLPCH4C58sIs55YkeX4+FwHgHxP+AMyVryR5cRZzP0hyYJ53AeBlhD8Ac+UPkhw9xcyzSf5Dn0xfgH0AOInwB2BO9Mn8nxz/GtLPTTPyXJKdSW5fqJ0A+AnhD8Cc6ZP5wyRbkjya46f7B5M8k+TpJP8xyb/03XsBhuEbeAEwp/pkvtQmsiPJFUlel+Ph/60+Oav7/wDME+EPwJw7cYf/gRNvACwCrvoAAEABwh8AAAoQ/gAAUIDwB2BRaxP5rTaRjwy9B8CoE/4ALHbXJ3nH0EsAjDpf1QeARa1P5leH3gHgdODEHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/nBCm8gZbSLLh94DAGA+CH/4iX+T5LNDLwEAMB/Gh14AFpHtSVYOvQQAwHwQ/nBCn8xTSZ4aeg8AgPngqg8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAlrvfegdAABgJGxe0fquf74wr9Xuzu7e++a5ep4TfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD+MkDaRK9pEzh56DwBg9Ah/GC3/Lcl7hl4CABg940MvAPxUrk7y9NBLAACjR/jDCOmT+dHQOwAAo8lVHwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAMCIaq2tbK39eWvt4dbantbaz083O76QiwEAAHPqD5Pc3Xt/b2vtjCRnTzco/AEAYAS11lYk+YUk/ypJeu8vJHlhunlXfQAAYDS9Mcn+JLe31u5rrd3WWjtnumHhDwAAi9Oq1tquk94+9LKPjyf5Z0n+c+/96iTPJvmd6R7mqg8AACxOB3rvm2f4+A+S/KD3/q0T7/95Zgh/J/4AADCCeu/7kny/tbbuxE/9YpKHppt34g8AAKPrt5LcceIr+jyW5F9PNyj8AQBgRPXe708y03Wgf+CqDwCcptpElraJbG8TefPQuwDDE/4AcPo6luOf/Pf00IsAw3PVBwBOU30yU0luGXoPYHFw4g8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoT/CGsTuaZN5HfbRNrQuwAAsLgJ/9H28SS3Jvm5oRcBAGBxGx96AX4mW5Jc1Cfz1NCLAACwuAn/EdYn8/dJ/n7oPQAAWPxc9QEAgAKEPwAAFCD8AQCgAOEPAAAFCH8AACjAV/UBAIDZujjJ9gV6rYvm9nFO/AEAoADhDwAABQh/AAAoQPgDAEABwv800Sbya20iNw29BwAAi5Ov6nP6uCfJGUMvAQDA4iT8TxN9MvuG3gEAgMXLVR8AAChA+AMAQAHCHwAAChD+AABQgPAHymsTWdYm8sah9wCA+ST8AZLfSHLL0EsAwHzy5TwBkj9K8mdDLwEA80n4A+X1ybyY5EdD7wEA88lVHwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEAB40MvAAAAvDqtte8lOZTkWJKjvffN080KfwAAGG3X9N4PnGrIVR8AAChA+AMAwOjqSf5na213a+1DMw266gMAAIvTqtbarpPe39Z73/aymX/Re3+8tfbaJF9trT3ce//6Kz1M+AMAwOJ0YKZP1k2S3vvjJ/77ZGttR5I3J3nF8HfVBwAARlBr7ZzW2vKXfpzkHUkmp5t34g8AAKPp/CQ7WmvJ8a7/Qu/97umGhT8AAIyg3vtjSa6c7byrPgAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwB5hve9uHs7dtG3oNAGoT/gDzrw29AACMD70AwGnvwv5fhl4BAJz4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAF+Dr+AAAwS48vvSC/e+FvLtCr/fs5fZoTfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAwIhqrS1prd3XWvvKqWaFPwAAjK7fTrJnNoPCHwAARlBr7fVJbkhy22zmhT8AAIymP0jy75JMzWZY+AMAwOK0qrW266S3D730gdbaryZ5sve+e7YPG5+XFQEAgJ/Vgd775mk+9pYk72qt/UqSs5Kc21r70977b0z3MCf+AAAwYnrvN/feX997vzjJjUn+cqboT4Q/AACU4KoPAACMsN77ziQ7TzXnxB8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFBA670PvQMAAIyENrG5565dC/NiG9vu3vvmuXqcE38AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAEZQa+2s1tq3W2sPtNYebK39/kzz4wu1GAAAMKeeT3Jt7/1wa21pkntba/+99/43rzQs/AEAYAT13nuSwyfeXXrirU8376oPAACMqNbaktba/UmeTPLV3vu3ppsV/gAAsDitaq3tOuntQy8f6L0f671fleT1Sd7cWpuY7mGu+gAAwOJ0oPe+eTaDvfenW2t/leSdSSZfacaJPwAAjKDW2urW2soTP35Nkl9K8vB08078AQBgNF2Y5E9aa0ty/ED/rt77V6YbFv4AADCCeu//K8nVs5131QcAAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFDA+NALAADAyHg0yfuGXuLVceIPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAYQa21N7TW/qq19lBr7cHW2m/PND++UIsBAABz6miSf9t7/05rbXmS3a21r/beH3qlYSf+AAAwgnrve3vv3znx40NJ9iR53XTzwh8AAEZca+3iJFcn+dZ0M676AADA4rSqtbbrpPe39d63vXyotbYsyReTfKT3/sx0DxP+AACwOB3ovW+eaaC1tjTHo/+O3vuXZpp11QcAAEZQa60l+aMke3rvnz/VvPAHAIDR9JYkW5Jc21q7/8Tbr0w37KoPAACMoN77vUnabOed+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4A1S1ty0ZegUAFo7wB6hob7s0yV9nb1s39CoALAzhD1DT3yX5T0n+duhFAFgY40MvAMAALuwvJrlz6DUAWDhO/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgALGh14AAABGxo9/mDz4+0Nv8ao48QcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AwCjZ25Zkb3vj0GswvNba9tbak621ydnMC38AgNHy7iSPDb0Ei8IfJ3nnbIeFPwDAaLk7yXVDL8Hweu9fT/LUbOfH53EXAADm2oX92SR/MfQajB7hDwAAi9Oq1tquk97f1nvf9mofJvwBAGBxOtB73zxXD3PHHwAAChD+AAAwglprdyb56yTrWms/aK395kzzrvoAAMAI6r3/+k8z78QfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAyd7Whl4BmF/CHwCq29t+Icnh7G1vGXoVYP4IfwDg8SRfT/LDoRcB5s/40AsAAAO7sD+a5JeHXgOYX078AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAJa733oHQAAYCS01u5OsmqBXu5A7/2dc/Uw4Q8AAAW46gMAAAUIfwAAKED4AwBAAcIfAAAKEP4AAFDA/wN1N1fTXdwpHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "node_size_factor = 1e2\n",
    "link_width_factor = 4\n",
    "\n",
    "nodes = list(G_ego.nodes())\n",
    "colors = pd.DataFrame(\n",
    "    [v[1][\"subject\"] for v in G_ego.nodes(data=True)], index=nodes, columns=[\"subject\"]\n",
    ")\n",
    "colors = np.argmax(target_encoding.transform(colors.to_dict(\"records\")), axis=1) + 1\n",
    "\n",
    "fig, ax = plt.subplots(1, 1, figsize=(15, 10))\n",
    "pos = nx.spring_layout(G_ego)\n",
    "# Draw ego as large and red\n",
    "node_sizes = [integrated_node_importance[graph_nodes.index(k)] for k in G_ego.nodes()]\n",
    "node_shapes = [\n",
    "    \"o\" if integrated_node_importance[graph_nodes.index(k)] > 0 else \"d\"\n",
    "    for k in G_ego.nodes()\n",
    "]\n",
    "positive_colors, negative_colors = [], []\n",
    "positive_node_sizes, negative_node_sizes = [], []\n",
    "positive_nodes, negative_nodes = [], []\n",
    "# node_size_sclae is used for better visualization of nodes\n",
    "node_size_scale = node_size_factor / np.max(node_sizes)\n",
    "for k in range(len(node_shapes)):\n",
    "    if list(nodes)[k] == target_nid:\n",
    "        continue\n",
    "    if node_shapes[k] == \"o\":\n",
    "        positive_colors.append(colors[k])\n",
    "        positive_nodes.append(list(nodes)[k])\n",
    "        positive_node_sizes.append(node_size_scale * node_sizes[k])\n",
    "\n",
    "    else:\n",
    "        negative_colors.append(colors[k])\n",
    "        negative_nodes.append(list(nodes)[k])\n",
    "        negative_node_sizes.append(node_size_scale * abs(node_sizes[k]))\n",
    "\n",
    "cmap = plt.get_cmap(\"jet\", np.max(colors) - np.min(colors) + 1)\n",
    "nc = nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=positive_nodes,\n",
    "    node_color=positive_colors,\n",
    "    cmap=cmap,\n",
    "    node_size=positive_node_sizes,\n",
    "    with_labels=False,\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    node_shape=\"o\",\n",
    ")\n",
    "nc = nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=negative_nodes,\n",
    "    node_color=negative_colors,\n",
    "    cmap=cmap,\n",
    "    node_size=negative_node_sizes,\n",
    "    with_labels=False,\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    node_shape=\"d\",\n",
    ")\n",
    "# Draw the target node as a large star colored by its true subject\n",
    "nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=[target_nid],\n",
    "    node_size=50 * abs(node_sizes[nodes.index(target_nid)]),\n",
    "    node_shape=\"*\",\n",
    "    node_color=[colors[nodes.index(target_nid)]],\n",
    "    cmap=cmap,\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    label=\"Target\",\n",
    ")\n",
    "\n",
    "edges = G_ego.edges()\n",
    "# link_width_scale is used for better visualization of links\n",
    "weights = [\n",
    "    integrate_link_importance[graph_nodes.index(u), list(Gnx.nodes()).index(v)]\n",
    "    for u, v in edges\n",
    "]\n",
    "link_width_scale = link_width_factor / np.max(weights)\n",
    "edge_colors = [\n",
    "    \"red\"\n",
    "    if integrate_link_importance[graph_nodes.index(u), list(Gnx.nodes()).index(v)] > 0\n",
    "    else \"blue\"\n",
    "    for u, v in edges\n",
    "]\n",
    "\n",
    "ec = nx.draw_networkx_edges(\n",
    "    G_ego, pos, edge_color=edge_colors, width=[link_width_scale * w for w in weights]\n",
    ")\n",
    "plt.legend()\n",
    "plt.colorbar(nc, ticks=np.arange(np.min(colors), np.max(colors) + 1))\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then remove the node or edge in the ego graph one by one and check how the prediction changes. By doing so, we can obtain the ground truth importance of the nodes and edges. Comparing the following figure and the above one can show the effectiveness of integrated gradients as the importance approximations are relatively consistent with the ground truth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAv4AAAIuCAYAAADHWwyWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAWvUlEQVR4nO3df6zdd13H8dfn9pb9aLvW2bIxIU520x/rHdts8RcgsA1BawgGxBFcUCTzL6NgEAmEIIvEQFnEaDTLGJo4lkyxxBidDrWRgbK0bJPDOs2YKCB3a4Pr2rXbuL0f/2indfbeXsa999zT9+OR3Kz3nne/993un2c//fTe1nsPAABwZhsb9gIAAMDiE/4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFjA97AQAAGBUTrfUjS/S5vpH8de/9NQv1POEPAADzdCTJLy7R53p/sn4hn+eqDwAAFCD8AQCgAOEPAAAFuOMPALCI9u7d+9zx8fGbk0zGoetMksH09PTbtm3b9siwl6lG+AMALKLx8fGbL7zwwi0bNmz4r7GxsT7sfYZpZmam7d+//9Kpqambk7x22PtUU/1PnQAAi21yw4YNj1WP/iQZGxvrGzZsOJjjf/vBEhP+AACLa0z0/68TvxcadAhc9QEAOINNTU2teMUrXrEpSQ4cOLBybGysn3/++dNJcu+99+47++yzF/wPJXfddde5U1NT4294wxseW+hn8+wJfwCAM9iFF1547IEHHrg/Sd7xjndctHr16mMf+MAHHp7vz5+ens74+LeXjHffffe5g8HgHOG/vPhrFgCAoq666qqJrVu3bpmYmNh64403rk+Sb33rW1mzZs0Vb33rW1+wcePGS3fv3r3q1ltvXXvxxRdPbt26dctb3vKWF1xzzTWXJMnBgwfHXv/611982WWXbdmyZculn/jEJ9YePny4ffjDH37erl27zt+8efOlH//4x79ruL9KnubEHwCgqNtuu+3fLrjggmOHDh0au+KKK7Zcd911/7Vu3bpjhw8fXvHyl7/80C233PLVQ4cOjU1MTEx+5jOfeWBiYuKpHTt2vPDpn/+ud73role/+tUHP/nJT35l//79K1784hdved3rXveld77znd8YDAbn3HLLLV8d5q+P/8uJPwBAUR/84Acv2LRp06Xbt2/f/PDDDz9n3759ZyXJypUr+3XXXfdoktxzzz1nv/CFL3xi48aNT42NjeXaa6/95tM/f/fu3eft3LnzeZs3b770ZS972aYnn3yyPfjgg88Z1q+HuTnxBwAo6FOf+tSaz33uc2v27t27b/Xq1X3btm2bjh49OpYkZ5111szY2OnPh3vv2bVr15e3bt365Mkf//SnP71mkdbmO+DEHwCgoEcffXTFunXrplevXt337Nlz9he/+MVVp5q78sorn3jooYfOfvDBB1fOzMzk9ttvP//p1175ylc+9pGPfOS5T7//2c9+9pwkWbNmzbHDhw/rzGXG/xAAgILe+MY3Hjx69OjYJZdcsvXd737397zoRS96/FRza9asmdm5c+d/XHPNNZsuu+yyLWvXrj123nnnHUuSD33oQ/955MiRsY0bN146MTGx9X3ve99FSbJjx45D999//7lbtmzxj3uXkda77ycBALBY7rvvvq9cfvnlB4a9x3fi4MGDY2vXrp2ZmZnJm9/85u+dnJw8+p73vOeRZ/u8++67b/3ll19+8QKuuGQuaq3/4hJ9rvcne3vv2xfqeU78AQCY086dOzds3rz50omJia1Hjx4de/vb375/2Dvx7fOPewEAmNMNN9zw8A033DDvb/rF8uTEHwAAChD+AACLa2ZmZqYNe4nl4sTvxcyw96hI+AMALK7B/v3714r/49G/f//+tUkGw96lInf8AQAW0fT09NumpqZunpqamoxD15kkg+np6bcNe5GKhD8AwCLatm3bI0leO+w9oPqfOgEAoAThDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAGEGttU2ttXtPenustfYrs82PL+VyAADAwui9/0uSK5KktbYiydeT7Jpt3ok/AACMvquTfLn3/u+zDQh/AAAYfdcmuW2uAeEPAADL0/rW2p6T3q4/1VBr7TlJXpvkT+Z6mDv+AACwPB3ovW+fx9yPJ/lC7/3huYac+AMAwGh7U05zzScR/gAAMLJaa6uSvCrJn51u1lUfAAAYUb33x5N893xmnfgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKaL33Ye8AAAAjYfva1vf8yNJ8rnZH9vbety/U85z4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoIDxYS8AsNTaZFYk2ZHkTUnWJvnXJDf1Qe4f6mIAsIiEP1BKm8yLkvx1klVJ1pz48KuSXN8m87dJfqYPcmRY+wHAYnHVByijTeb7knwmyQX53+hPjh+CnJPk6iR/3ibThrAeACwq4Q9U8v4cP+mfLezPSfKDSX50qRYCgKUi/IES2mRWJfnpJCtOM7oqyTsWfyMAWFrCH6ji+Umm5zHXkkwu8i4AsOSEP1DFUzn9af/JswBwRhH+QBX/nuSxecw9meTPF3kXAFhywh8ooQ8yk+TG5LRfqrMn+b3F3wgAlpbwByr5aJIvJDk6y+tHkry9D/IfS7cSACwN4Q+U0Qd5Ksk1SX43yeEcv/pzMMnjSb6c5M19kD8Y3oYAsHh8516glD7Ik0l+rU3mfUlekmR1jt//v68P0oe6HAAsIuEPlNQHeSLJ3w57DwBYKq76AABAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AABhRrbV1rbU/ba090Frb11r74dlmfQMvAAAYXR9Nckfv/Q2tteckOXe2QeEPAAAjqLW2NsmPJvm5JOm9P5XkqdnmXfUBAIDR9H1J9if5eGvtntbaza21VbMNC38AAFie1rfW9pz0dv0zXh9P8v1Jfr/3fmWSx5P8+mwPc9UHAACWpwO99+1zvP61JF/rvX/+xPt/mjnC34k/AACMoN77VJKvttY2nfjQ1Unun23eiT8AAIyuX0py64mv6PNQkp+fbVD4AwDAiOq935tkrutA/8NVHwAAKED4AwBAAcIfAAAKcMcfKK9N5pIkP5TkaJI7+yCHhrwSACw44Q+U1SazLsntSV6aZDpJT7KyTeaGJL/VB+nD3A8AFpLwB0pqk2lJ7kxyWZKznvHye3P8ux/+zlLvBQCLxR1/oKqXJtmc/x/9SXJukve3yaxc2pUAYPEIf6CqHUlWzfH6eJLJJdoFABad8Aeqaqd5vc9jBgBGhvAHqrojx+/xz6YnGSzRLgCw6IQ/UNXuJA8leeoUrz2e5Df74JSvAcBIEv5ASSe+VOfVSe5OciTHY/9QkieSfDTJzuFtBwALz5fzBMrqgxxI8rI2mckkP5zj38DrL/sg3xzuZgCw8IQ/UF4fZBD3+QE4w7nqAwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChgf9gIAADAyLk5yyxJ9rosW9nFO/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKCA8WEvAAAAPDutta8kOZTkWJLp3vv22WaFPwAAjLZX9t4PnG7IVR8AAChA+AMAwOjqSf6mtba3tXb9XIOu+gAAwPK0vrW256T3b+q93/SMmZf23r/eWntukjtbaw/03v/hVA8T/gAAsDwdmOsf6yZJ7/3rJ/77SGttV5IfSHLK8HfVBwAARlBrbVVrbc3TP07yY0kGs8078QcAgNF0QZJdrbXkeNd/ovd+x2zDwh8AAEZQ7/2hJJfPd95VHwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoYHzYCwBQW5vMqiQvTjLVB3lg2PsAnKmc+AMwNG0yP5Xk4SSfSrK3TWb3iT8IALDAhD8AQ9Emsz7JrUlWJVmb5NwkP5TkhmHuBXCmEv4ADMuPJZl+xsfOSvIzQ9gF4Iwn/AEYlseS9FN8/PBSLwJQgfAHYFj+JsmRJDMnfexIkg8PZx2AM5vwB2Ao+iBPJXlJkt1JjiU5kOS9ST42xLUAzli+nCcAQ9MHeSjJ1cPeA6ACJ/4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIACxoe9AAAAjIqvr7ww733eLyzRZ/vNBX2aE38AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAMCIaq2taK3d01r7i9PNCn8AABhdv5xk33wGhT8AAIyg1trzk+xIcvN85oU/AACMpt9O8mtJZuYzLPwBAGB5Wt9a23PS2/VPv9Ba+8kkj/Te9873YeOLsiIAAPCdOtB73z7Lay9J8trW2k8kOTvJea21P+69/+xsD3PiDwAAI6b3/u7e+/N77xcnuTbJ380V/YnwBwCAElz1AQCAEdZ7351k9+nmnPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKaL33Ye8AAAAjoU1u77l9z9J8sq1tb+99+0I9zok/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAjqLV2dmvt7tbafa21L7XWfmOu+fGlWgwAAFhQTya5qvd+uLW2MsldrbW/6r3/06mGhT8AAIyg3ntPcvjEuytPvPXZ5l31AQCAEdVaW9FauzfJI0nu7L1/frZZ4Q8AAMvT+tbanpPern/mQO/9WO/9iiTPT/IDrbXJ2R7mqg8AACxPB3rv2+cz2Ht/tLX290lek2Rwqhkn/gAAMIJaaxtaa+tO/PicJK9K8sBs8078AQBgND0vyR+11lbk+IH+7b33v5htWPgDAMAI6r3/c5Ir5zvvqg8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKCA8WEvAAAAI+PLSd447CWeHSf+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAEdRae0Fr7e9ba/e31r7UWvvluebHl2oxAABgQU0n+dXe+xdaa2uS7G2t3dl7v/9Uw078AQBgBPXev9F7/8KJHx9Ksi/J98w2L/wBAGDEtdYuTnJlks/PNuOqDwAALE/rW2t7Tnr/pt77Tc8caq2tTvLJJL/Se39stocJfwAAWJ4O9N63zzXQWluZ49F/a+/9z+aaddUHAABGUGutJflYkn299xtPNy/8AQBgNL0kyXVJrmqt3Xvi7SdmG3bVBwAARlDv/a4kbb7zTvwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChgfNgLAADAyHjiP5Mv/cawt3hWnPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAABGUGvtltbaI621wXzmhT8AAIymP0zymvkOC38AABhBvfd/SPLN+c4LfwAAKGB82AsAAACntL61tuek92/qvd/0bB8m/AEAYHk60HvfvlAPc9UHAAAKEP4AADCCWmu3JfnHJJtaa19rrf3CXPOu+gAAwAjqvb/p25l34g8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AAChA+AMAQAHCHwAAChD+AABQgPAHAIAChD8AABQg/AEAoADhDwAABQh/AAAoQPgDAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAUIPwBAKAA4Q8AAAUIfwAAKED4AwBAAcIfAAAKEP4AAFCA8AcAgAKEPwAAFCD8AQCgAOEPAAAFCH8AACig9d6HvQMAAIyE1todSdYv0ac70Ht/zUI9TPgDAEABrvoAAEABwh8AAAoQ/gAAUIDwBwCAAoQ/AAAU8N848VE7h824qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x720 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "[X, _, A], y_true_all = all_gen[0]\n",
    "N = A.shape[-1]\n",
    "X_bk = deepcopy(X)\n",
    "edges = [(graph_nodes.index(u), graph_nodes.index(v)) for u, v in G_ego.edges()]\n",
    "nodes = [list(Gnx.nodes()).index(v) for v in G_ego.nodes()]\n",
    "selected_nodes = np.array([[target_idx]], dtype=\"int32\")\n",
    "clean_prediction = model.predict([X, selected_nodes, A]).squeeze()\n",
    "predict_label = np.argmax(clean_prediction)\n",
    "groud_truth_edge_importance = np.zeros((N, N), dtype=\"float\")\n",
    "groud_truth_node_importance = []\n",
    "\n",
    "for node in nodes:\n",
    "    if node == target_idx:\n",
    "        groud_truth_node_importance.append(0)\n",
    "        continue\n",
    "    X = deepcopy(X_bk)\n",
    "    # we set all the features of the node to zero to check the ground truth node importance.\n",
    "    X[0, node, :] = 0\n",
    "    predict_after_perturb = model.predict([X, selected_nodes, A]).squeeze()\n",
    "    prediction_change = (\n",
    "        clean_prediction[predict_label] - predict_after_perturb[predict_label]\n",
    "    )\n",
    "    groud_truth_node_importance.append(prediction_change)\n",
    "\n",
    "node_shapes = [\n",
    "    \"o\" if groud_truth_node_importance[k] > 0 else \"d\" for k in range(len(nodes))\n",
    "]\n",
    "positive_colors, negative_colors = [], []\n",
    "positive_node_sizes, negative_node_sizes = [], []\n",
    "positive_nodes, negative_nodes = [], []\n",
    "# node_size_scale is used for better visulization of nodes\n",
    "node_size_scale = node_size_factor / max(groud_truth_node_importance)\n",
    "\n",
    "for k in range(len(node_shapes)):\n",
    "    if nodes[k] == target_idx:\n",
    "        continue\n",
    "    if node_shapes[k] == \"o\":\n",
    "        positive_colors.append(colors[k])\n",
    "        positive_nodes.append(graph_nodes[nodes[k]])\n",
    "        positive_node_sizes.append(node_size_scale * groud_truth_node_importance[k])\n",
    "    else:\n",
    "        negative_colors.append(colors[k])\n",
    "        negative_nodes.append(graph_nodes[nodes[k]])\n",
    "        negative_node_sizes.append(node_size_scale * abs(groud_truth_node_importance[k]))\n",
    "\n",
    "X = deepcopy(X_bk)\n",
    "for edge in edges:\n",
    "    original_val = A[0, edge[0], edge[1]]\n",
    "    if original_val == 0:\n",
    "        continue\n",
    "    # we set the weight of a given edge to zero to check the ground truth link importance\n",
    "    A[0, edge[0], edge[1]] = 0\n",
    "    predict_after_perturb = model.predict([X, selected_nodes, A]).squeeze()\n",
    "    groud_truth_edge_importance[edge[0], edge[1]] = (\n",
    "        predict_after_perturb[predict_label] - clean_prediction[predict_label]\n",
    "    ) / (0 - 1)\n",
    "    A[0, edge[0], edge[1]] = original_val\n",
    "#     print(groud_truth_edge_importance[edge[0], edge[1]])\n",
    "\n",
    "fig, ax = plt.subplots(1, 1, figsize=(15, 10))\n",
    "cmap = plt.get_cmap(\"jet\", np.max(colors) - np.min(colors) + 1)\n",
    "# Draw the target node as a large star colored by its true subject\n",
    "nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=[target_nid],\n",
    "    node_size=50 * abs(node_sizes[nodes.index(target_idx)]),\n",
    "    node_color=[colors[nodes.index(target_idx)]],\n",
    "    cmap=cmap,\n",
    "    node_shape=\"*\",\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    label=\"Target\",\n",
    ")\n",
    "# Draw the ego net\n",
    "nc = nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=positive_nodes,\n",
    "    node_color=positive_colors,\n",
    "    cmap=cmap,\n",
    "    node_size=positive_node_sizes,\n",
    "    with_labels=False,\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    node_shape=\"o\",\n",
    ")\n",
    "nc = nx.draw_networkx_nodes(\n",
    "    G_ego,\n",
    "    pos,\n",
    "    nodelist=negative_nodes,\n",
    "    node_color=negative_colors,\n",
    "    cmap=cmap,\n",
    "    node_size=negative_node_sizes,\n",
    "    with_labels=False,\n",
    "    vmin=np.min(colors) - 0.5,\n",
    "    vmax=np.max(colors) + 0.5,\n",
    "    node_shape=\"d\",\n",
    ")\n",
    "edges = G_ego.edges()\n",
    "# link_width_scale is used for better visulization of links\n",
    "link_width_scale = link_width_factor / np.max(groud_truth_edge_importance)\n",
    "weights = [\n",
    "    link_width_scale\n",
    "    * groud_truth_edge_importance[graph_nodes.index(u), list(Gnx.nodes()).index(v)]\n",
    "    for u, v in edges\n",
    "]\n",
    "\n",
    "edge_colors = [\n",
    "    \"red\"\n",
    "    if groud_truth_edge_importance[graph_nodes.index(u), list(Gnx.nodes()).index(v)] > 0\n",
    "    else \"blue\"\n",
    "    for u, v in edges\n",
    "]\n",
    "\n",
    "ec = nx.draw_networkx_edges(G_ego, pos, edge_color=edge_colors, width=weights)\n",
    "plt.legend()\n",
    "plt.colorbar(nc, ticks=np.arange(np.min(colors), np.max(colors) + 1))\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
